WO2015050380A1 - Appareil d'affichage d'image et son procédé de pointage - Google Patents

Appareil d'affichage d'image et son procédé de pointage Download PDF

Info

Publication number
WO2015050380A1
WO2015050380A1 PCT/KR2014/009273 KR2014009273W WO2015050380A1 WO 2015050380 A1 WO2015050380 A1 WO 2015050380A1 KR 2014009273 W KR2014009273 W KR 2014009273W WO 2015050380 A1 WO2015050380 A1 WO 2015050380A1
Authority
WO
WIPO (PCT)
Prior art keywords
remote control
angle
pointing
display
camera
Prior art date
Application number
PCT/KR2014/009273
Other languages
English (en)
Korean (ko)
Inventor
오쿠노호장
히라사와굉우
무라카미에드워드
최상언
Original Assignee
삼성전자 주식회사
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from JP2013207599A external-priority patent/JP6200756B2/ja
Priority claimed from JP2014173247A external-priority patent/JP6370165B2/ja
Application filed by 삼성전자 주식회사 filed Critical 삼성전자 주식회사
Priority to EP14850816.1A priority Critical patent/EP3054693B1/fr
Priority to US15/027,108 priority patent/US9910507B2/en
Priority claimed from KR1020140132316A external-priority patent/KR102207256B1/ko
Publication of WO2015050380A1 publication Critical patent/WO2015050380A1/fr

Links

Images

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N21/00—Selective content distribution, e.g. interactive television or video on demand [VOD]
    • H04N21/40—Client devices specifically adapted for the reception of or interaction with content, e.g. set-top-box [STB]; Operations thereof
    • H04N21/41—Structure of client; Structure of client peripherals
    • H04N21/422—Input-only peripherals, i.e. input devices connected to specially adapted client devices, e.g. global positioning system [GPS]
    • H04N21/42204—User interfaces specially adapted for controlling a client device through a remote control device; Remote control devices therefor
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/0304—Detection arrangements using opto-electronic means
    • G06F3/0308—Detection arrangements using opto-electronic means comprising a plurality of distinctive and separately oriented light emitters or reflectors associated to the pointing device, e.g. remote cursor controller with distinct and separately oriented LEDs at the tip whose radiations are captured by a photo-detector associated to the screen
    • G—PHYSICS
    • G06—COMPUTING; CALCULATING OR COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a 3D space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors

Definitions

  • the present invention relates to an image display apparatus for controlling a pointer to be displayed on a display and a pointing method thereof.
  • a television remote control having a pointing function may be used to click a link or the like.
  • Such a remote controller includes a motion sensor such as an accelerometer or an angular velocity sensor, and is configured to detect an amount of change in the attitude angle of the remote control from these sensor outputs and convert the pointing angle calculated from the amount of change in the attitude angle into a pointer position on the display (patent) See Document 1).
  • a motion sensor such as an accelerometer or an angular velocity sensor
  • the pointer position on the display is calculated cumulatively based on the posture angle change amount of the remote controller, it is called a relative pointing method.
  • the direction indicated by the remote controller and the pointer position on the display are greatly shifted, so that the user may not obtain a natural feeling.
  • This phenomenon is caused by a change in the relative positional relationship between the display and the remote controller or by accumulating the attitude error of the remote controller estimated from the motion sensor.
  • the relative pointing remote controller 200 having an angular velocity sensor has a pointer position on the screen by changing left and right the indicating direction PD, which is a direction indicated by the front end of the remote controller 200. Move left and right.
  • the reference direction SD is set in the remote controller 200, and the pointer position on the display 100 is determined according to a pointing angle which is an angle formed by the instruction direction PD with respect to the reference direction SD.
  • the operating angle range CR is set to a predetermined solid angle centered on the reference direction SD with a predetermined correspondence between the pointing range on the display 100 shown in FIG. For example, when the operation angle range CR is set small, the pointer is sensitive to the movement of the remote controller 200, and when the operation angle range CR is set large, the pointer movement is insensitive.
  • the pointer when the reference direction SD and the pointing direction PD coincide with each other, the pointer is disposed at the center of the screen, and when there is a pointing angle, the pointer position on the display 200 according to the set operating angle range CR. Is changed.
  • P1 in FIG. 35 shows a case where the remote controller 200 is in front of the display, and the reference direction SD, which is the center of the operating angle range CR, is directed perpendicularly to the screen center PO.
  • the reference direction SD coincides with the direction perpendicular to the center of the screen as described above, the user may obtain a natural feeling because the direction PD of the remote controller 200 and the pointer position coincide well.
  • the reference direction SD is in a state toward the right end of the display.
  • the instruction direction PD of the remote controller 200 is directed toward the right end of the display 100 to coincide with the reference direction SD, the pointer is set to the right end of the screen by setting in P1 of FIG. 34. It will be displayed in the center of the screen.
  • the instruction position PD of the remote controller 200 and the pointer position are greatly shifted, it is unnatural to the user.
  • the conventional pointing device forcibly coincides the pointer position with the instruction direction of the remote controller by for example pressing the reset button after the user directs the instruction direction of the remote controller toward the screen center. Or, it is configured to stop when the pointer moves to the end of the display, and the user can adjust the direction of the remote control to match the pointer position on the screen.
  • Patent Documents 2 to 4 various techniques for moving the display position of the pointer on the display following the movement of the remote controller have also been proposed (for example, see Patent Documents 2 to 4).
  • the technique of Patent Document 2 when the optical pointing device is moved, the light receiving device moves the display position of the pointer according to the position signal obtained by calculating the change in the light receiving signal.
  • Patent Literature 3 a cursor for comparing the coordinates of a captured cursor image in a captured image captured by a camera of a remote controller with the coordinates of a display cursor image displayed at that time and displaying it at a remote control instruction position on the detected display screen. The image is being generated.
  • Patent Document 1 Japanese Patent Application Laid-Open No. 2002-62981
  • Patent Document 2 Japanese Patent Application Laid-Open No. 2006-155345
  • Patent Document 3 Japanese Patent Application Laid-Open No. 2008-181198
  • Patent Document 4 Japanese Patent Application Laid-Open No. 2008-227605
  • any pointing device when the instruction direction of the remote control and the pointer position on the display are displaced, the user has to perform the resetting operation of the reference direction as described above every time to impair the user's feeling.
  • the present invention has been made in view of the above-described problems, and an object thereof is to automatically adjust so that the direction of instruction of the remote control and the pointer position on the screen are almost identical. That is, in this embodiment, even if the position of a remote control is changed, a pointing system is realized in which the center direction of the operating angle range of the remote control is directed toward the center of the screen without requiring user's operation.
  • Another object of the present invention is to provide a camera on the display side in the case where the instruction direction of the remote control and the pointer position on the screen almost coincide.
  • the pointing device of the present invention it becomes possible to automatically adjust so that the direction of instruction of the remote control and the pointer position on the screen are almost identical.
  • FIG 1 is an overall configuration diagram of an image display system according to an embodiment of the present invention.
  • FIG. 2 is a view showing the appearance of a remote controller used in the embodiment of the present invention.
  • 3 and 4 are diagrams for explaining the outline of the operation of the embodiment of the present invention.
  • FIG. 5 is a functional block diagram of an image display device according to an embodiment of the present invention.
  • FIG. 6 is a functional block diagram of a remote control device according to an embodiment of the present invention.
  • Fig. 7 is a functional block diagram of the image display system in the first embodiment according to the present invention.
  • FIG. 10 is a flowchart of a pointing method of the image display device according to the embodiment of the present invention.
  • FIG. 11 is a flowchart of a remote control control method of the remote control device according to the embodiment of the present invention.
  • FIG. 12 is a functional block diagram of an image display system according to a second embodiment according to the present invention.
  • Fig. 13 is a functional block diagram of the image display system in the third embodiment according to the present invention.
  • FIG. 14 is a functional block diagram of an image display system in a fourth embodiment according to the present invention.
  • 20 to 22 are diagrams for explaining the operation of the calibration unit in the image display device of the fourth embodiment according to the present invention.
  • Fig. 23 is an overall operation flowchart of the image display device of the fourth embodiment according to the present invention.
  • FIG. 25 is an example of a graph that defines a conversion rule when the conversion rule from the pointing angle to the pointing coordinate position changes continuously in accordance with the distance from the image display device to the remote control.
  • Fig. 28 is a diagram showing a functional configuration example of an image display device and a remote controller that constitute the image display system according to the fifth embodiment of the present invention.
  • Fig. 31 is a schematic perspective view showing an LED imaged when the remote controller faces the screen of the image display device according to the fifth embodiment of the present invention.
  • FIG. 32 is a schematic perspective view showing an LED imaged when the remote controller rotates upward in the horizontal axis direction with respect to the apparatus screen of the image display according to the fifth embodiment of the present invention.
  • Fig. 33 is a schematic diagram showing LEDs photographed when the remote controller rotates upward in the horizontal axis direction and counterclockwise in the vertical axis direction with respect to the screen of the image display device according to the fifth embodiment of the present invention. Perspective view.
  • 35 is a schematic diagram illustrating the pointing operation characteristic in the conventional image display apparatus.
  • the present invention provides a display, a reference angle detector for detecting a reference angle which is an angle formed by the instruction direction of the remote control relative to the reference direction set for the display, and a remote control direction which is a direction when the remote control is viewed from a reference point set on the display
  • a remote control direction detection unit for detecting a reference direction and a reference direction control unit for adjusting the reference direction so that the reference point on the display is viewed from the remote control based on the reference angle and the remote control direction
  • a controller configured to control a pointer to be displayed at a corresponding position on the display based on a pointing angle corresponding to an angle formed by an instruction direction of the remote controller with respect to the reference direction.
  • the reference point on the display when the reference direction set from the remote control is seen from the remote control means that the reference point is located on a virtual straight line extending from the remote control in the reference direction.
  • the concept also includes a state in which the reference point exists near a straight line. In other words, it refers to a state in which the reference point exists in the virtual cylinder extending in the reference direction from the remote controller with a predetermined radius.
  • the reference direction adjusting unit may be configured to adjust the reference direction every predetermined time.
  • the reference angle detector is further configured based on an azimuth sensor provided in the remote control, a reference azimuth set by the azimuth sensor as an azimuth indicating a reference direction, and a remote azimuth azimuth using a direction azimuth indicating a direction indicated by the remote control. It may be provided with a reference angle calculation unit for calculating the angle.
  • the image display apparatus may further include an angular velocity sensor installed in the remote controller, a pointing angle calculator configured to calculate the pointing angle based on a reference pointing angle and an angular velocity output from the angular velocity sensor, and the reference direction adjusting unit
  • the pointing angle calculation unit may be configured to set an angle formed by the instructing direction with respect to a direction as the reference pointing angle.
  • the pointing angle calculation unit stores the remote control direction when the reference direction has been adjusted in the past, and is configured to calculate the pointing angle when a difference between the past remote control direction and the current remote control direction is greater than or equal to a predetermined threshold. May be used.
  • the image display device may further include a pointing angle calculator configured to calculate the pointing angle based on a reference pointing angle, and the reference direction adjusting unit uses the reference pointing angle as the reference pointing angle. It may be comprised so that it may be set to a pointing angle calculation part.
  • the remote control direction detecting unit calculates a remote control direction angle which is an angle formed by a camera provided to capture the remote control with respect to the display and the remote control direction with respect to a direction perpendicular to the display based on the captured image of the camera.
  • the remote control direction angle calculation part may be provided.
  • the camera is installed to have a photographing direction parallel to a specific vertical plane perpendicular to the display at a horizontal edge of the display, and the remote control direction angle calculating unit is mounted on the display based on the captured image of the camera.
  • a first remote control direction angle calculating unit configured to calculate a first remote control direction angle, which is an angle formed between a vertical direction and a direction obtained by projecting the remote control direction on a horizontal plane; an angle formed by the captured image of the camera and the display and the shooting direction A second angle formed by a direction perpendicular to the display and a direction obtained by projecting the remote control direction onto the specific vertical surface, based on a photographing angle, a distance from the camera to the remote control, and an edge length in the vertical direction of the display; Second remote control direction to calculate the remote control direction angle An angle calculating unit, wherein the reference direction adjusting unit is configured to provide the reference point on the display when the reference direction is viewed from the remote control based on the reference angle, the first remote control direction angle, and the second remote control direction angle.
  • the second remote control direction angle calculating unit calculates a photographing remote control angle formed by the photographing direction and the direction from the camera to the remote controller based on the image of the camera, and the difference between the photographing remote control angle and the photographing angle and the The second remote control direction angle may be calculated based on the distance and the length.
  • the pointing device calculates the photographing angle based on an image obtained by photographing the remote control with the camera provided to photograph the horizontal direction, and an image obtained by photographing the remote control with the camera installed to have the photographing direction.
  • the photographing angle calculator may be further provided.
  • the camera may be provided to have a downward shooting direction in the center of the upper edge of the display.
  • the camera is installed to have a photographing direction parallel to a horizontal plane on the vertical edge of the display, the remote control direction angle calculation unit based on the captured image of the camera and the direction perpendicular to the display and the remote control direction
  • a first remote control direction angle calculating unit for calculating a first remote control direction angle, which is an angle formed by a projection obtained by projecting a specific vertical plane perpendicular to the display, and an angle formed by the photographed image of the camera and the display and the shooting direction.
  • a second remote control direction angle that is an angle formed by a direction perpendicular to the display and a direction obtained by projecting the remote control direction on a horizontal plane based on a photographing angle, a distance from the camera to the remote control, and a border length in a horizontal direction of the display; Second remote control direction angle calculation And the reference direction adjusting unit adjusts the reference direction so that the reference point on the display is viewed when the reference direction is viewed from the remote control based on the reference angle, the first remote control direction angle, and the second remote control direction angle. It may be configured to adjust.
  • the second remote control direction angle calculating unit calculates a photographing remote control angle formed by the photographing direction and the direction from the camera to the remote controller based on the image of the camera, and the difference between the photographing remote control angle and the photographing angle and the The second remote control direction angle may be calculated based on the distance and the length.
  • the image display device is based on an image obtained by photographing the remote control with the camera provided to photograph a direction parallel to the specific vertical plane, and an image obtained by photographing the remote control with the camera provided to have the photographing direction.
  • the apparatus may further include a photographing angle calculator which calculates the photographing angle.
  • the controller may include a position calculator configured to calculate a corresponding position on the display based on the pointing angle, and a display controller configured to control a pointer to be displayed at the position, wherein the position calculator includes the display from the display to the remote controller.
  • the longer the distance may be configured to calculate the position such that the distance from the reference point to the position relative to the unit angle of the pointing angle becomes longer.
  • the present invention provides a camera capable of capturing a plurality of light sources of the remote control for the display;
  • a remote control attitude estimation unit for estimating a remote control attitude at the time of imaging corresponding to a direction at the time of imaging of the remote control according to the number or positions of light sources picked up by the camera;
  • a calibration unit for calibrating the reference attitude of the remote control corresponding to the reference direction of the remote control to the attitude of the remote control during imaging estimated by the remote control attitude estimation unit;
  • a pointing position estimator for estimating a pointing position according to the calibrated reference posture and an amount of posture change measured by a motion sensor installed in the remote controller.
  • the remote control posture estimating unit includes: a correspondence relationship storage unit for storing a correspondence relationship between the number and position of the light sources in the image and the posture at the time of image capturing; And an imaging posture output section for outputting an imaging posture corresponding to the number and position of the light sources among the images picked up by the camera, with reference to the correspondence relationship storage section.
  • the present invention also provides a method for detecting a reference angle, which is an angle formed by an instruction direction of a remote controller with respect to a reference direction set for a display, and detecting a remote controller direction which is a direction when the remote controller is viewed from a reference point set on the display. And adjusting the reference direction so that the reference point on the display is viewed from the remote control based on the reference angle and the remote control direction, and the reference remote control relative to the reference direction.
  • a pointing method is also provided that includes controlling a pointer to be displayed at a corresponding position on the display based on a pointing angle according to an angle formed by a direction indicated by.
  • the present invention includes the steps of estimating the remote control posture at the time of imaging corresponding to the direction at the time of imaging of the remote control according to the number or position of the light source of the remote control captured by the camera installed for the display; Calibrating the reference attitude of the remote controller corresponding to the reference direction of the remote controller to the attitude of the remote controller during the estimated imaging; Estimating a pointing position according to the calibrated reference posture and a posture change amount measured by a motion sensor installed in the remote controller; And displaying an object at the pointing position on the pointing object according to the estimated pointing position.
  • the estimating the remote control posture during the imaging may include: storing a correspondence relationship between the number and position of the light sources in the image and the posture during the imaging when the image is captured; And outputting a posture at the time of imaging corresponding to the number and position of the light source among the images picked up by the camera with reference to the stored correspondence.
  • the present invention includes a light emitting device unit having at least one light source; A direction sensor unit for sensing a remote control azimuth value indicating a direction indicated by the remote control device; An operation unit for operating the pointing of the image display device; And a control unit which turns on the light emitting element unit according to a pointing operation of the operation unit and modulates a remote control azimuth value sensed by the orientation sensor unit into a remote control code and transmits the remote control code to the image display device through the light emitting element unit.
  • the remote control azimuth value sent to the device provides the remote control device, characterized in that the image display device is used to determine a pointing position on the display of the image display device.
  • the present invention comprises the steps of turning on the light emitting element according to the pointing operation input and sensing a remote control azimuth value indicating the direction indicated by the remote control device; Converting the sensed remote control azimuth value into a remote control code; And outputting the remote control code to the image display device through a light emitting element, wherein the remote control orientation value transmitted to the image display device determines that the image display device determines a pointing position on the display of the image display device. It provides a remote control method characterized in that used to.
  • the present invention provides a function of detecting a reference angle which is an angle formed by the instruction direction of the remote controller with respect to a reference direction set for the display on a computer, and a remote control direction which is a direction when the remote control is viewed from a reference point set on the display.
  • a function of detecting, a function of adjusting the reference direction so that the reference point on the display is viewed from the remote control based on the reference angle and the remote control direction, and the reference direction Also provided is a program for realizing a function of controlling a pointer to be displayed at a corresponding position on the display based on a pointing angle corresponding to an angle made by the direction indicated by the remote control.
  • the image display system 300 includes a television 100 corresponding to an embodiment of an image display apparatus, an angular velocity sensor 21, and an orientation sensor together with a function for operating the television 100.
  • the remote control 200 provided with 22 is provided.
  • the television 100 is an example of an image display apparatus.
  • the television 100 is an apparatus that receives broadcast contents including an image and an audio transmitted from a broadcasting station, displays an image on a screen, and simultaneously outputs an audio.
  • the television 100 also displays on the screen an image for performing operations such as selecting broadcast content.
  • the remote controller 200 is used to perform operations such as selecting broadcast content on an image displayed on the screen of the television 100.
  • a camera 11 for capturing the remote control 200 is provided at an upper center portion of the television 100, and the camera 11 is provided at the front end of the remote control 200 as shown in FIG. 2.
  • An LED marker 23 is provided to easily grasp the position of the remote controller 200.
  • the LED marker 23 is described as a visible light LED.
  • the camera 11 is demonstrated as being a visible light color camera.
  • the image display device is not limited to a television, but may be used in a PC, a signage monitor, or the like.
  • an image display apparatus means a liquid crystal monitor, an organic EL monitor, etc.
  • the image display system 300 measures the pointing angle A2, which is an angle formed by the instruction direction PD of the remote controller 200 with respect to the reference direction SD set in the remote controller 200, at the angular velocity. It calculates with the output from the sensor 21 and the said orientation sensor 22. In addition, the pointing device 300 accumulates rotational conversion with respect to the pointing angle A2, and indicates that the pointing direction PD of the remote controller 200 is pointing on the screen of the television 100. Is configured to display a pointer P on the screen.
  • FIG. 3 shows the positional relationship between the television 100 and the remote control 200 from above
  • FIG. 4 shows the positional relationship between the television 100 and the remote control 200 from the side.
  • the remote control 200 has an operating angle range CR having a predetermined solid angle with a virtual straight line in the reference direction SD as a central axis.
  • the relationship between the CR) and the pointing range of the television 100 is set. That is, the sensitivity of the pointer P to the posture change of the remote controller 200 is adjusted by adjusting the size of the manipulation angle range CR.
  • the image display system 300 will be described with reference to Figs.
  • the central axis of the operating angle range CR which is the reference direction SD, always faces the direction of the screen center PO of the television 100.
  • indication direction PD of the remote control 200 always keeps the point which is pointing on the screen of the television 100 and the position of pointer P almost consistent with the position of the remote control 200.
  • a detailed configuration for realizing such an operation will be described below.
  • FIG. 5 is a functional block diagram of an image display device according to an embodiment of the present invention.
  • the image display device 100 of FIG. 5 includes a reference angle detector 4, a remote control direction detector 3, a reference direction controller 14, a controller 5, and a display 18.
  • the control unit 5 also includes a position calculating unit 16 and a display control unit 17 which are disclosed later in the description of the image display system according to the first to fourth embodiments.
  • the reference angle detector 4 detects a reference angle AC, which is an angle formed by the instruction direction COM of the remote controller 200 with respect to the reference direction CO set for the display.
  • the direction COM of the remote controller 200 is detected from the orientation sensor 22 installed in the remote controller 200.
  • the reference angle detector 4 is a reference orientation set to an orientation value output by the orientation sensor 22 installed in the remote controller 200 when the direction COM of the remote controller 200 is perpendicular to the display.
  • the reference angle AC is calculated based on the remote control orientation COM value set as the orientation value output from the orientation sensor 22 in correspondence with the instruction direction COM of the remote controller 200.
  • the remote control direction detecting unit 3 detects the remote control direction RD which is the direction when the remote control 200 is viewed from the reference point PO set on the display. At this time, the remote control direction RD is detected from an image of the remote control captured by the camera 110 installed in the image display apparatus 100.
  • the reference direction controller 14 When the reference direction controller 14 sees the reference direction SD set in the remote controller from the remote controller 200, the reference direction controller 14 is configured so that the reference angle detector 4 has a reference point PO on the display of the image display apparatus 100.
  • the reference direction SD is adjusted based on the detected reference angle AC and the remote control direction RD detected by the remote control direction detection unit 3.
  • the controller 5 controls the pointer P to be displayed at a corresponding pointing position on the display of the image display apparatus 100 based on the pointing angle according to the angle formed by the instruction direction of the remote controller with respect to the reference direction SD.
  • the direction of the remote control 200 by using the direction information 22 output from the orientation sensor 22 installed in the remote control 200 and the camera 110 installed in the image display device 100 (COM) and the position of the pointer (P) on the screen can be automatically adjusted to almost match.
  • FIG. 6 is a functional block diagram of a remote control device according to an embodiment of the present invention.
  • the light emitting element unit 26 includes an LED marker 23 to emit light with at least one light source.
  • the at least one light source may be configured in a predetermined arrangement pattern.
  • the orientation sensor unit 22 senses a remote controller orientation value indicating a direction indicated by the remote controller 200.
  • An angular velocity sensor 210 provided as an option senses an angular velocity value corresponding to the movement of the remote controller.
  • the angular velocity sensor 210 may use a gyroscope as an example.
  • the operation unit 25 operates the pointing of the image display device.
  • the control unit 24 turns on at least one light source of the light emitting element unit 230 according to a pointing operation of the operation unit 25, and / or each speed sensor (or the remote controller azimuth value sensed by the orientation sensor unit 22).
  • Each speed value sensed by 210 is modulated by a remote control code and output to the image display apparatus 100 through the light emitting element unit 230.
  • the remote control azimuth value transmitted to the image display apparatus 100 is used by the image display apparatus 100 to determine the pointing position on the display of the image display apparatus.
  • the configuration diagram of the image display system 300 of the first embodiment is as shown in FIG. 7, and the remote control direction detection unit 3, the reference angle detection unit 4, the reference direction adjustment unit 14, and the pointing angle calculation are performed. It is comprised by the part 15, the coordinate calculation part 16, the display control part 17, and the display part 18. As shown in FIG. For each part, the function is realized by executing a program for the pointing device stored in the memory in an arithmetic mechanism (computer) composed of a CPU, a memory, an input / output means, and the like in the television 100.
  • arithmetic mechanism computer
  • the pointing device 300 of the present invention can adjust the reference direction SD toward the screen center PO in any of the left and right directions and the vertical direction.
  • the remote control direction detecting unit 3 is an example of a remote control direction detecting unit, which is a direction when the remote control unit 200 is viewed from a screen center PO, which is a reference point set in the television 100, as shown in FIG. 8.
  • This remote control direction RD can also be defined by the virtual straight line which connects the said screen center PO and the front-end
  • standard which this virtual straight line extends are said The direction SD continues to coincide always.
  • the remote control direction detection mechanism 3 is directed toward the user from the television 100, and includes a camera 11 for photographing the LED marker 23 of the remote control 200, and the camera ( A value indicating the remote control direction RD based on the camera image PIC captured by 11), and the remote control direction angle AP which is an angle formed by the remote control direction RD with respect to the direction perpendicular to the screen center PO. It is comprised by the remote control direction angle calculation part 12 which calculates ().
  • the remote control direction angle calculator 12 binarizes the camera image PIC shown in FIG. 8A as shown in FIG. 8B, and binarizes the LED marker 23 only. Create an image.
  • the remote control direction angle calculator 12 calculates the separation distance DET from the center line of the television 100 in the binary image to the detection position of the LED marker 23.
  • the remote control direction angle calculating unit 12 may calculate the remote control direction angle AP from the separation distance DET, and as a result, may specify the remote control direction RD.
  • LED marker 23 may be detected based on the brightness of the LED marker 23 in 1st Embodiment, you may detect based on the color, blinking pattern, etc. of the LED marker 23, for example.
  • a reflective marker or the like may be used instead of the LED marker 23.
  • one or two or more LED markers 23 may be used, and the shape of the LED marker 23 may be freely determined.
  • FIG. 9 the operation angle range CR is omitted for clarity and only the reference direction SD is shown. However, the operation angle range CR shown in FIGS. 3, 4, and 8 is actually shown in FIG. 9. Sets the reference direction SD as the central axis.
  • the reference angle detection mechanism 4 is an example of a reference angle detection unit, and is a reference angle AC which is an angle formed by the instruction direction PD of the remote controller 200 with respect to the reference direction CO set for the television 100. Will be detected.
  • the reference direction CO is set in a direction perpendicular to the television 100, and the direction of the PD PD of the remote control 200 is several degrees with respect to the reference direction CO.
  • the reference angle detector 4 detects whether it is inclined. That is, the reference angle AC is expressed as an absolute value based on the reference direction CO with respect to how much the direction of instructing PD of the remote controller 200 is inclined with respect to the screen of the television 100. will be.
  • the reference angle detector 4 includes the orientation sensor 22 and the reference angle calculator 13 as shown in FIG. 7.
  • the azimuth sensor 22 outputs the azimuth of the instruction direction PD, which is a direction indicated by the tip of the remote controller 200, as a numerical value.
  • the azimuth sensor 22 outputs when the direction PD of the remote controller 200 is directed toward the reference direction CO perpendicular to the screen of the television 100.
  • the bearing which the bearing sensor 22 currently outputs in correspondence with the reference bearing CO and the direction in which the remote controller 200 is instructed is also called the remote controller bearing COM.
  • the reference bearing CO arranges the remote control 200 in a direction perpendicular to the screen after starting the television 100, and reads the value of the remote control bearing COM output by the orientation sensor 22 at that time in advance. Can be set.
  • the television 100 may not be set manually after startup, but may be automatically set in advance, such as by providing an orientation sensor in the television 100.
  • the reference angle calculator 13 calculates the reference angle AC shown in FIG. 9 from the difference between the reference orientation CO and the remote controller orientation COM.
  • the reference angle AC is a value based on the azimuth sensor 22, the reference angle AC is a value that is determined solely for the instruction direction PD of the remote controller 200.
  • the reference direction SD is determined by the reference direction adjusting unit 14 based on the remote control direction RD and the reference angle AC respectively detected by the remote control direction detecting unit 3 and the reference angle detecting unit 4. Will be described.
  • the reference direction adjustment unit 14 operates to adjust the reference direction SD such that the reference direction SD which is the center of the manipulation angle range CR coincides with the remote control direction RD.
  • the direction PD of the remote controller 200 indicates the screen center PO
  • the direction of the reference direction SD is appropriately changed so that the reference direction SD and the remote control direction RD coincide with each other. I'm doing it.
  • the reference direction controller 14 adjusts the remote control direction RD with respect to the reference pointing angle A1 used as an initial value in the pointing angle calculator 15 described later, as shown in FIG. 7.
  • the reference direction SD is calculated from the remote control direction angle AP and the reference angle AC, and the reference direction SD is adjusted to face the screen center PO.
  • the reference pointing angle A1 is a value indicating how many degrees the current indicating direction PD of the remote controller 200 is inclined with respect to the case where the indicating direction PD of the remote controller 200 indicates the screen center PO. to be. Accordingly, as shown in FIG. 9, the reference pointing angle A1 may be obtained by subtracting the remote control direction angle AP from the reference angle AC. This is evident from the relationship between the line perpendicular to the screen and the reference direction CO at the center of the screen PO in parallel. In addition, the reference pointing angle A1 may also be referred to as an accurate value detected by the remote control direction detection unit 3 and the reference angle detection unit 4 with reference to the reference direction CO as a reference.
  • the pointing angle calculator 15 calculates the current pointing angle A2 based on the reference pointing angle A1 and the angular velocity output from the angular velocity sensor 21. That is, the pointing angle calculator 15 uses the reference pointing angle A1 as an initial value, and repeats the rotational conversion cumulatively from the angular velocity obtained by the angular velocity sensor 21 from the current pointing angle ( It is configured to calculate A2). In addition, the pointing angle calculator 15 holds the pointing angle A2 that the current pointing angle A2 is replaced with the reference pointing angle A1 every predetermined time by the reference direction adjusting unit 14. The reference direction SD continues to face the screen center PO.
  • the reference pointing angle A1 used in the pointing angle calculator 15 is calculated based on the output of the azimuth sensor 22, and the angular velocity sensor 21 is applied to the current pointing angle A2 which is sequentially calculated. ) Is based on output.
  • the angular velocity sensor 21 may be, for example, a gyroscope.
  • the coordinate calculator 16 is an example of a position calculator that calculates a corresponding position on the display, and calculates a corresponding pointing coordinate position POS based on the pointing angle A2.
  • the display control unit 17 outputs an operation screen image DP in which the pointer P is superimposed on a basic screen such as a menu screen based on the pointing coordinate position POS.
  • a basic screen such as a menu screen based on the pointing coordinate position POS.
  • the part which consists of the said coordinate calculation part 16 and the said display control part 17 is an example of the control part which controls a pointer to be displayed in the corresponding position on a display.
  • the display unit 18 displays the operation screen image DP on the television 100.
  • the absolute remote control direction RD detected by the said remote control direction detection part 3 and the said reference angle detection part 4 and the absolute direction based on the television 100 are referred to.
  • the reference angle AC which is a value
  • the current reference pointing angle A1 when the reference direction SD of the remote controller 200 faces the screen center PO may be calculated.
  • the reference pointing angle A1 is replaced with the current pointing angle A2 used in the pointing angle calculator 15 so that the reference direction SD is toward the screen center PO.
  • the operation can be reset automatically.
  • the current pointing angle A2 held by the pointing angle calculator 15 is calculated by the reference direction controller 14.
  • the point in which the instruction direction PD of the remote controller 200 is pointing on the screen can almost coincide with the position of the pointer P on the screen.
  • the reference direction controller 14 calculates the reference pointing angle A1 every predetermined time, and updates the pointing angle A2 held by the pointing angle calculator 15. As a result, the reference direction is adjusted. (SD) is adjusted to face the center of the screen (PO) at regular intervals.
  • the user can operate the pointer P with a natural operating feeling by the remote controller 200.
  • FIG. 10 is a flowchart of a pointing method of the image display device according to the embodiment of the present invention.
  • the image display apparatus 100 detects a reference angle AC, which is an angle formed by the instruction direction COM of the remote controller 200 with respect to the reference direction CO set for the display. At this time, the direction COM of the remote controller 200 is detected from the orientation sensor 22 installed in the remote controller 200.
  • the image display apparatus 100 detects the remote control direction RD, which is the direction when the remote controller 200 is viewed from the reference point PO set on the display of the image display apparatus 100. At this time, the remote control direction RD is detected from an image of the remote control captured by the camera 110 installed in the image display apparatus 100.
  • the reference angle AC may be such that there is a reference point PO on the display of the image display apparatus 100.
  • the reference direction SD based on the remote control direction RD.
  • the image display apparatus 100 moves the pointer P to a corresponding pointing position on the display of the image display apparatus 100 based on a pointing angle according to an angle formed by the direction indicated by the remote controller with respect to the reference direction SD. Control to display.
  • the direction of the remote control 200 by using the direction information 22 output from the orientation sensor 22 installed in the remote control 200 and the camera 110 installed in the image display device 100 (COM) and the position of the pointer (P) on the screen can be automatically adjusted to almost match.
  • FIG. 11 is a flowchart of a remote control control method of the remote control device according to the embodiment of the present invention.
  • the remote controller 200 checks whether a pointing menu operation is performed.
  • the remote control device 200 turns on the light emitting device according to a pointing manipulation input and senses a remote control orientation value indicating a direction indicated by the remote control device.
  • the remote control apparatus 200 communicates a command according to another key manipulation.
  • the remote controller 200 converts the sensed remote controller azimuth value into a remote controller code.
  • a remote control code is output to the image display device through a light emitting device.
  • the remote control azimuth value transmitted to the image display device 100 is used by the image display device 100 to determine the pointing position on the display of the image display device.
  • the image display system 300 of 2nd Embodiment has the said pointing angle calculation part with respect to the remote control direction angle AP output from the said remote control direction detection part 3.
  • the difference is that (15) is configured to be acquired.
  • the pointing angle calculator 15 stores a past remote control direction angle AP when a substitution with the reference pointing angle A1 output from the reference direction adjustment unit 14 is performed.
  • the pointing angle calculator 15 compares the new remote control direction angle AP output from the remote control direction detection unit 3 with the past remote control direction angle AP to determine whether the pointing angle calculation unit 15 has a difference greater than or equal to a predetermined value. It is configured to permit replacing the pointing angle A2 with the reference pointing angle A1.
  • the relative position of the remote controller 200 with respect to the television 100 changes, for example, as shown in FIG. 3, and from the remote controller direction detection mechanism 3.
  • the reference direction SD can be adjusted only when a change of more than a predetermined value occurs in the output remote control direction angle AP.
  • the reference direction SD can be adjusted to ensure that they continue to coincide. have.
  • the angular velocity sensor 21 is omitted in the image display system 300 of the third embodiment, and the pointing angle calculator 15 is the reference pointing angle.
  • the difference is configured to calculate the pointing angle A2 based on (A1). More specifically, the pointing device 300 according to the third embodiment periodically calculates the remote control direction angle AP output from the remote control direction detection unit 3 and the reference angle AC output from the reference angle detection unit 4.
  • the pointing angle calculator 15 is configured to output the reference pointing angle A1 as it is as a pointing angle A2.
  • the reference sensor SD is always directed toward the screen center PO while reducing the cost by using a sensor provided in the remote controller 200 so that the reference direction PD and the pointer P of the remote controller 200 are always maintained. You can keep the positions consistent.
  • FIG. 14 is a functional block diagram of an image display system in a fourth embodiment according to the present invention.
  • the television 100 has a camera 11 as described above.
  • the first remote control detector 120, the second remote controller detector 121, the calibration unit 122, the first pointing angle calculator 140, the second pointing angle calculator 141, and the pointing An angle update unit 150, a coordinate calculation unit 16, a display control unit 17, and a display unit 18 are provided.
  • the camera 11 generates a camera image PIC by capturing the scenery on the screen side (viewer side) of the television 100, and generates the camera image PIC from the first remote controller detector 120 and the second remote controller detector 121. )
  • the camera 11 is provided as an example of a photography means.
  • the first remote control detection unit 120 detects an LED marker mounted on the remote control 200 from the camera image PIC output by the camera 11. Then, the remote control direction angle APh, which is an angle formed by a horizontal component (component projected onto the horizontal plane) in the direction perpendicular to the screen of the television 100 and the direction from the screen center PO to the remote control 200, is calculated. Output to the 1 pointing angle calculation unit 140.
  • the remote control direction angle APh is used as an example of the first remote control direction angle, which is an angle formed by the direction perpendicular to the display and the direction obtained by projecting the remote control direction onto one surface, and the first remote control direction angle is used.
  • the first remote control detector 120 is provided as an example of the first remote control direction angle calculator for calculating the. The operation of the first remote control detector 120 will be described later in detail.
  • the second remote control detection unit 121 detects an LED marker mounted on the remote control 200 from the camera image PIC output by the camera 11.
  • the angle formed by the vertical component (the component projected on the vertical plane passing through the screen center PO and perpendicular to the screen) in the direction perpendicular to the screen of the television 100 and the direction from the screen center PO to the remote controller 200.
  • the remote controller direction angle APv is calculated and output to the second pointing angle calculator 141.
  • the second remote control detector 121 uses the photographing angle APbase formed by the screen of the television 100 and the photographing direction of the camera 11.
  • the remote control direction angle APv is used as an example of the second remote control direction angle, which is an angle formed by the direction perpendicular to the display and the direction obtained by projecting the remote control direction onto the other surface.
  • the second remote control detection unit 121 is provided as an example of the second remote control direction angle calculation unit to calculate. The operation of the second remote control detector 121 will also be described in detail later.
  • a remote control direction detection part the part which consists of the camera 11, the 1st remote control detection part 120, and the 2nd remote control detection part 121 is provided.
  • the calibration unit 122 calculates and sets the photographing angle APbase used in the second remote control detector 121 in advance.
  • the calibration part 122 is provided as an example of a photographing angle calculation part. The operation of the calibration unit 122 will also be described later in detail.
  • the first pointing angle calculator 140 calculates a reference pointing angle A1h based on the remote control direction APh and the remote control direction COMh that is a horizontal component of the direction COM output by the remote control unit 200. Output to the pointing angle update unit 150. The operation of the first pointing angle calculator 140 will also be described in detail later.
  • the second pointing angle calculator 141 calculates a reference pointing angle A1v based on the remote control direction APv and the remote control direction COMv that is a vertical component of the direction COM output by the remote control unit 200. Output to the pointing angle update unit 150.
  • the operation of the second pointing angle calculator 141 will also be described later in detail.
  • the part which consists of the 1st pointing angle calculation part 140 and the 2nd pointing angle calculation part 141 is provided as an example of a reference angle detection part and a reference direction adjustment part.
  • the pointing angle updating unit 150 may include an angular velocity GYR output by the remote controller 200, a reference pointing angle A1h output by the first pointing angle calculator 140, and a second pointing angle calculator 141.
  • the pointing angle A2 is calculated on the basis of the reference pointing angle A1 obtained by combining the output reference pointing angle A1v, and output to the coordinate calculating unit 16. Basically, the currently held pointing angle A2 is cumulatively rotated based on the angular velocity GYR to update the new pointing angle A2. Then, at a predetermined timing, a reset operation is performed in which the value of the pointing angle A2 is replaced with the value of the reference pointing angle A1.
  • the rotation angle is repeatedly accumulated based on the angular velocity GYR to update the pointing angle A2.
  • the timing which the operation position of a user changed for example can be considered.
  • the change in the user's operation position may be determined from the change in the user's image position in the camera image PIC, for example.
  • the reset operation is not limited to this timing, but may be performed periodically, for example at regular time intervals. Even if the user's operation position is changed in this way, the pointing angle update unit 150 matches the direction of the center of the operating angle range of the remote controller 200 with the direction of the remote controller 200 toward the screen center PO.
  • the pointing angle update part 150 is provided as an example of the pointing angle calculation part which calculates a pointing angle.
  • the coordinate calculation unit 16 calculates the pointing coordinate position POS on the screen of the television 100 based on the pointing angle A2 and outputs it to the display control unit 17.
  • the coordinate calculating part 16 is provided as an example of the position calculating part which calculates the corresponding position on a display.
  • the display control unit 17 generates the operation screen image DP in which the pointer is superimposed on the menu screen based on the pointing coordinate position POS and outputs it to the display unit 18.
  • the part which consists of the coordinate calculation part 16 and the display control part 17 is provided as an example of the control part which controls to display a pointer in the corresponding position on a display.
  • the display unit 18 displays the operation screen image DP on the screen of the television 100.
  • the display part 18 is provided as an example of the display means which displays an image on a display screen.
  • the remote controller 200 includes an angular velocity sensor 21, an acceleration sensor, and a geomagnetic sensor 220.
  • the angular velocity sensor 21 is a gyroscope which detects the angular velocity of an object, for example by detecting the inertial force applied to a rotating object.
  • the angular velocity GYR of the remote controller 200 is detected and output to the pointing angle updater 150.
  • the acceleration sensor and the geomagnetic sensor 220 use an acceleration sensor that detects acceleration by detecting a change in capacitance caused by a slight movement of the small movable portion, and, for example, the direction of the geomagnetic field using a magnetic sensor element. It is an attitude sensor including a geomagnetic sensor to detect.
  • the orientation COM of the remote controller 200 is detected based on the attitude of the remote controller 200, and the remote component orientation COMh, which is a horizontal component thereof, is output to the first pointing angle calculator 140, and the vertical component thereof.
  • the remote control direction COMv is output to the second pointing angle calculator 141.
  • FIG. 15 is a diagram for describing an operation of the first remote control detector 120.
  • FIG. 15A shows a camera image PIC.
  • the first remote control detection unit 120 performs a binarization process on the camera image PIC with a predetermined threshold to obtain a binarized image as shown in Fig. 15B.
  • the 1st remote control detection part 120 detects the area
  • FIG. 15C is a diagram showing the positional relationship between the television 100 and the remote controller 200 from above, and is a remote controller in the direction of the angle APh with reference to the direction perpendicular to the screen of the television 100. (200) is present.
  • the first remote control detection unit 120 is determined from the detection position DETh.
  • the remote control direction angle APh is calculated. Specifically, the remote control direction angle APh corresponding to the length from the center of the image to the detection position DETh is calculated by using the length from the left end to the right end of the image in FIG. 15B corresponding to the angle of view.
  • LED was detected based on the brightness of LED in the above, LED may be detected based on the color of LED, or LED may be detected based on the blinking pattern of LED.
  • a reflective marker may be used instead of the LED marker.
  • one or two or more LED markers 23 may be used, and the shape of the LED marker 23 may be freely determined.
  • FIG. 16 is a view showing the positional relationship between the television 100 and the remote controller 200 as in FIG.
  • the operation angle range is abbreviate
  • the vector COMh is the remote control orientation output from the acceleration and the geomagnetic sensor 220.
  • the vector COh is a reference direction perpendicular to the screen of the television 100.
  • the remote control direction angle when viewed from the television 100 is APh.
  • the remote controller 200 is directed toward the center of the screen PO of the television 100 as the center of the operating angle range of the remote controller 200.
  • the direction rotated by the angle APh from the reference direction COh may be the center direction of the operating angle range of the remote controller 200.
  • the remote controller 200 faces the direction rotated by the angle APh from the reference direction COh, the remote controller 200 indicates the direction of the screen center PO.
  • the first pointing angle calculator 140 obtains an angle ACh formed by the remote control direction COMh and the reference direction COh, and connects the remote control 200 with the screen center PO and the direction of the remote control (
  • the reference pointing angle A1h formed by COMh is obtained by subtracting the angle APh from the angle ACh.
  • the vector COh arranges the remote controller 200 in a direction perpendicular to the screen, and reads the value of the remote controller orientation COMh outputted by the acceleration sensor and the geomagnetic sensor 220 at that time.
  • This can be set in advance.
  • the television 100 may not be set manually after startup, but may be automatically set in advance, such as by providing an orientation sensor in the television 100.
  • the vector COh has been described as a direction perpendicular to the screen of the television 100, the direction CO2 may be any direction, such as a direction parallel to the screen of the television 100.
  • the operation of the second remote control detector 121 will be described in detail. Although the above-described first remote control detector 120 performs the processing in the horizontal direction, the second remote control detector 121 performs the same processing in the vertical direction. 17 is a diagram for explaining an operation of the second remote control detector 121.
  • FIG. 17A shows a camera image PIC.
  • the second remote control detection unit 121 performs a binarization process on the camera image PIC at a predetermined threshold to obtain a binarized image as shown in Fig. 17B.
  • the 2nd remote control detection part 121 detects the area
  • FIG. 17C is a view showing the positional relationship between the television 100 and the remote controller 200 from a side view, and in the direction of the angle APv with reference to the direction perpendicular to the screen of the television 100. (200) is present.
  • the second remote control detection unit 121 is determined from the detection position DETv.
  • the remote control direction angle APv is calculated. Specifically, the remote control direction angle APv corresponding to the length from the center of the image to the detection position DETv is calculated by using the length from the top to the bottom of the image in FIG. 17B corresponding to the angle of view.
  • the first remote control detector 120 is based on the premise that the camera 11 is mounted at the center of the screen in the horizontal direction
  • the second remote control detector 121 is the same as the camera 11 in the center of the screen even in the vertical direction. Is assumed to be fitted.
  • the camera 11 is mounted on the screen center PO, it is necessary to modify the operation of the second remote control detector 121 by mounting the camera 11 on the upper or lower frame of the television 100. have.
  • the camera 11 is mounted at the center of the upper frame of the television 100.
  • 1 also shows an example in which the camera 11 is mounted at the center of the upper frame of the television 100.
  • the operation of the second remote control detector 121 when the camera 11 is mounted on the upper portion of the television 100 will be described in detail.
  • 18 is a view showing the positional relationship between the television 100 and the remote controller 200 from a side view, and the remote controller 200 in the direction of the angle APv based on the direction perpendicular to the screen of the television 100. It shows what exists.
  • the operation angle range is abbreviate
  • the second remote control detector 121 moves the remote controller from the detection position DETv based on only the relationship between the detection position DETv and the remote control direction angle APv determined according to the characteristics of the camera 11. Although the angle APv is obtained, the following correction is necessary to obtain the remote control direction angle APv from the detection position DETv when the camera 11 is mounted on the upper portion of the television 100.
  • the second remote control detector 121 obtains the distance L1 from the camera 11 to the remote controller 200.
  • the distance L1 may be obtained by referring to this lookup table.
  • the interval D between the two LED markers 23 on the camera image PIC is inversely proportional to the distance L1, and thus the interval D
  • the relationship between the distance and the distance L1 may be held in the lookup table, and the distance L1 may be obtained by referring to the lookup table.
  • APbase is an angle (hereinafter, referred to as "shooting angle") that the photographing direction of the camera 11 forms with the screen.
  • shooting angle an angle (hereinafter, referred to as "shooting angle" that the photographing direction of the camera 11 forms with the screen.
  • the photographing angle becomes a lowered angle.
  • APbefore is an angle (shooting remote control angle) which the photographing direction of the camera 11 and the direction from the camera 11 to the remote control 200 makes, and is calculated
  • FIG. 19 is a view showing the positional relationship between the television 100 and the remote controller 200 from the side view as in FIG. 17C.
  • the operation angle range is abbreviate
  • the vector COMv is a remote controller orientation output from the acceleration and the geomagnetic sensor 220.
  • the vector COv is a reference direction perpendicular to the screen of the television 100.
  • the remote control direction angle when viewed from the television 100 is APv.
  • the remote controller 200 is directed toward the center of the screen PO of the television 100 as the center of the operating angle range of the remote controller 200.
  • the direction rotated by the angle APv from the reference direction COh may be the center direction of the operating angle range of the remote controller 200.
  • the remote controller 200 faces the direction rotated by the angle APv from the reference direction COh, the remote controller 200 indicates the direction of the screen center PO.
  • the second pointing angle calculator 141 obtains an angle ACv formed by the remote control direction COMv and the reference direction COh, and connects the remote control 200 with the screen center PO and the remote control direction (
  • the reference pointing angle A1v formed by COMv is obtained by subtracting the angle APv from the angle ACv.
  • the vector COv arranges the remote control 200 in a direction perpendicular to the screen after starting the television 100, and reads the value of the remote control orientation COMv output by the acceleration sensor and the geomagnetic sensor 220 at that time.
  • This can be set in advance.
  • the television 100 may not be set manually after startup, but may be automatically set in advance, such as by providing an orientation sensor in the television 100.
  • the vector COv has been described as a direction perpendicular to the screen of the television 100.
  • the vector COv may be another direction, such as a direction parallel to the screen of the television 100, since the direction may be a reference direction.
  • the calibration unit 122 uses the angle of view ⁇ in the vertical direction of the camera 11 and the movement amount ⁇ D of the remote controller 200 on the image as shown in FIG. 22.
  • N is the number of pixels corresponding to the vertical direction angle of view.
  • the vertical direction angle of view may be calculated from the size of the image sensor and the focal length of the camera 11 or may be obtained from an actual value.
  • the measured value is a result of measuring the vertical direction angle of view experimentally using the camera 11, and may be configured to store this.
  • the method of calculating the imaging angle of the camera 11 using the movement amount (D) of the remote control 200 on an image was shown above, you may use the background movement amount on an image.
  • the movement amount ⁇ D of the background image can be obtained.
  • the photographing angle of can be calculated.
  • the sensor which measures the angle can be measured directly by attaching the encoder to the angle adjusting portion of the camera 11.
  • the camera 11 and the LED marker 23 were demonstrated as what is a visible light camera and a visible light LED so far, it is not limited to this.
  • an infrared camera and an infrared LED may be sufficient.
  • FIG. 23 is a flowchart illustrating an overall operation of the television 100.
  • the camera 11 first, the camera 11 generates a camera image PIC by photographing the screen side (viewer side) landscape of the television 100 (step 301). Then, the first remote control detector 120 based on the camera image (PIC) generated in step 301, the horizontal component in the direction perpendicular to the screen of the television 100 and the direction from the screen center PO to the remote controller 200.
  • the remote control direction angle APh which is an angle formed by this, is calculated (step 302), and the second remote control detector 121 captures the camera image PIC generated in step 301, the screen of the television 100, and the camera 11.
  • the remote control direction angle APv which is an angle formed by the direction perpendicular to the screen of the television 100 and the vertical component of the direction from the screen center PO to the remote control 200 is calculated ( Step 303).
  • the first pointing angle calculator 140 calculates a reference pointing angle based on the remote control direction APh calculated in step 302 and the remote control orientation COMh, which is a horizontal component of the instruction direction output by the remote control 200.
  • A1h) step 304
  • the second pointing angle calculator 141 controls the remote control direction COMv which is a vertical component of the remote control direction angle APv calculated in step 303 and the direction indicated by the remote control 200.
  • the reference pointing angle A1v is calculated based on (step 305).
  • the pointing angle updating unit 150 combines the angular velocity GYR output by the remote controller 200, the reference pointing angle A1h calculated in step 304, and the reference pointing angle A1v calculated in step 305.
  • the pointing angle A2 is calculated based on the angle A1 (step 306).
  • the coordinate calculation unit 16 calculates the pointing coordinate position POS on the screen of the television 100 based on the pointing angle A2 (step 307). Then, the display control unit 17 generates the operation screen image DP in which the pointer is superimposed on the menu screen based on the pointing coordinate position POS calculated in step 307 (step 308). Then, the display unit 18 displays the operation screen image DP generated in step 308 on the screen of the television 100.
  • the camera 11 is mounted on the upper or lower frame of the television 100, and the first remote control detector 120 and the first pointing angle calculator 140 perform processing on the horizontal plane.
  • the second remote control detector 121 and the second pointing angle calculator 141 perform the processing on the vertical plane perpendicular to the screen of the television 100.
  • the camera 11 is mounted on the left or right frame of the television 100, and the first remote control detector 120 and the first pointing angle calculator 140 are perpendicular to the screen of the television 100.
  • the processing may be performed on the image processing, and the second remote control detection unit 121 and the second pointing angle calculation unit 141 may perform the processing on the horizontal plane.
  • the horizontal plane is an example of one surface
  • the vertical plane perpendicular to the screen of the television 100 is an example of another surface.
  • vertical to the screen of the television 100 is one surface
  • an horizontal surface is an other surface.
  • the camera 11 is mounted in the center of the frame of the television 100.
  • the center of the frame is ideally the frame center, it does not necessarily need to be the frame center. This is because the deviation from the frame midpoint of the mounting position of the camera 11 can be corrected on the camera image PIC generated by the camera 11 photographing the viewer side.
  • the distance L1 between the camera 11 and the remote controller 200 is measured based on the area S of the LED marker 23 and the interval between the LED markers 23 in the camera image PIC.
  • other methods may be used.
  • a stereo camera may be used.
  • a stereo camera is a camera that can obtain depth information (distance) to an object by using parallax information of two cameras arranged left, right, or up and down. And a distance image which shows the distance to the object in front of a camera as an image can be obtained.
  • a stereo camera can be comprised by providing one camera 110 further in the center of the lower side. Since the coordinates of the remote controller 200 are known from the camera image PIC from the camera 11 already equipped, the distance L1 between the camera 11 and the remote controller 200 is referred to by referring to the data of the same coordinates in the distance image. ) Can be measured.
  • a distance image sensor may be used. Since this is also a sensor for obtaining a distance image, the distance L1 between the camera 11 and the remote control 200 can be measured similarly.
  • a distance image sensor has a time of flight (TOF) method using a difference in arrival time of light, and a structure that irradiates infrared light having a specific pattern shape and calculates a distance to an object based on the distortion of the pattern shape.
  • TOF time of flight
  • an ultrasonic wave may be used.
  • An ultrasonic transmitter is provided in the remote controller 200, two ultrasonic receivers are provided at different positions of the television 100, and ultrasonic waves are transmitted from the ultrasonic transmitters of the remote controller 200 so that the phase difference of the ultrasonic waves received by the two ultrasonic receivers is provided.
  • the distance L1 between the camera 11 and the remote controller 200 can be measured from the triangulation principle based on the distance between the two ultrasonic receivers.
  • three or more ultrasonic receivers may be sufficient, and when three or more ultrasonic receivers are used, a more accurate measurement is possible.
  • a radio wave may be used.
  • the remote controller 200 is provided with a radio wave transmitter and two radio wave receivers at different positions of the television 100, and transmits radio waves from the remote control unit 200 so that the radio waves are received from the phase difference of the radio waves received by the two radio wave receivers. From the triangulation principle, the distance L1 between the camera 11 and the remote control 200 can be measured. In addition, three or more radio wave receivers may be sufficient, and when it is three or more, the measurement of a higher precision is attained.
  • this 4th Embodiment was supposed to have a structure based on 1st Embodiment, you may have a structure based on 2nd Embodiment or 3rd Embodiment. That is, as in the second embodiment, the pointing angle update unit 150 acquires the remote control direction angle APh and the remote control direction angle APv from the first remote control detector 120 and the second remote control detector 121. It is good also as a structure which does not provide the angular velocity sensor 21 similarly to 3rd Embodiment.
  • FIG. 24 is a diagram illustrating a functional configuration example of the position calculator 16 in the image display device 100. That is, embodiment of this invention corresponds to the position calculation part 16 in each embodiment of 4th embodiment from 1st Embodiment mentioned above.
  • the position calculator 16 includes a conversion table storage unit 161, a distance acquisition unit 162, a coordinate determination unit 163, and a coordinate output unit 164.
  • the conversion table storage unit 161 stores a conversion table that is referred to when converting the pointing angle A2 into the pointing coordinate position POS.
  • This conversion table is a table which defines what coordinate position is calculated as the pointing coordinate position POS when the pointing angle A2 is an angle.
  • a plurality of conversion tables are stored corresponding to the distance from the television 100 to the remote control 200. For example, the long distance conversion table when the distance from the television 100 to the remote control 200 is DR or more, and the short distance conversion table when the distance from the television 100 to the remote control 200 is DR or less. do.
  • the distance acquisition unit 162 acquires the distance from the television 100 to the remote control 200.
  • the remote control direction angle calculation unit 12 calculates the remote control direction angle AP
  • the method similar to that of the second remote control detection unit 121 in the fourth embodiment is performed. Since the distance from the camera 11 to the remote control 200 can also be calculated, this distance may be acquired as the distance from the television 100 to the remote control 200.
  • the second remote control detection unit 121 calculates the distance L1 from the camera 11 to the remote control 200, this distance is the distance from the television 100 to the remote control 200.
  • the distance from the camera 11 to the remote control 200 is not acquired as it is from the television 100 to the remote control 200 as it is, but is based on the distance from the camera 11 to the remote control 200.
  • the distance from the screen center PO to the remote controller 200 may be calculated, and the distance may be acquired as the distance from the television 100 to the remote controller 200.
  • the coordinate determination unit 163 reads out the corresponding conversion table from the conversion table storage unit 161 according to the distance acquired by the distance acquisition unit 162. And using this conversion table, the pointing coordinate position POS which corresponds to the pointing angle A2 is determined. For example, if the distance acquired by the distance acquisition unit 162 is DR or more, the pointing coordinate position POS is determined using the long distance conversion table, and when the distance acquired by the distance acquisition unit 162 is below DR, The pointing coordinate position (POS) is determined using the conversion table.
  • the coordinate output unit 164 outputs the pointing coordinate position POS determined by the coordinate determination unit 163 to the display control unit 17.
  • the coordinate determination unit 163 selects any one of the plurality of conversion tables stored in the conversion table storage unit 161 according to the distance acquired by the distance acquisition unit 162 and uses the pointing angle A2. Is converted into a pointing coordinate position (POS). That is, the conversion rule from the pointing angle A2 to the pointing coordinate position POS is changed in stages according to the distance from the television 100 to the remote controller 200. However, such a conversion rule may be changed continuously. 25 is an example of a graph that defines such a continuously changing conversion rule.
  • the horizontal axis represents the distance from the television 100 to the remote control 200
  • the vertical axis represents the number of pixels (distance) from the screen center PO to the pointing coordinate position POS per unit angle of the pointing angle A2. Equivalent to). For example, assume that the distance acquired from the distance acquisition unit 162 is DRX. Then, as shown, the coordinate determination unit 163 acquires (PX / ⁇ ) indicating that the number of pixels per unit angle ⁇ is PX. The coordinate determination unit 163 calculates the pointing coordinate position POS based on the number of pixels obtained by multiplying PX / ⁇ by the pointing angle A2.
  • the reference direction is set in a direction perpendicular to the screen of the television, but may be set to be inclined with respect to the screen as shown in FIG. 26, for example. Even in such a case, the reference pointing angle can be calculated if the inclination angle of the reference direction with respect to the television screen is already known, and the reference direction can be adjusted. In addition, the reference direction may be set parallel to the screen.
  • the image display system 300 of this embodiment is the television 100 which is a pointing object, the camera 13 installed in the said television 100, and the remote control for operating the television 100 ( 200, and is used for the purpose of the user pointing the remote control 200 to any point on the television screen.
  • this image display system 300 is provided with the motion sensor called the acceleration sensor 32 and the angular velocity sensor 33 in the said remote control 200, and the instruction
  • PL indication direction
  • the pointing device 100 is configured to display and move an object at a point on the television screen from a measurement value to a pointing point estimated to be currently indicated by the remote controller 200.
  • the image display system 300 of this embodiment is what is called a relative pointing system
  • posture attitude of the said remote control 200 used as the basis of estimation of a pointing position is performed automatically, and calibration start is started. The user does not need to perform a special operation.
  • the physical configuration of the image display system 300 includes the remote controller 200, the camera 11, and a calculation mechanism E added to the television 100.
  • the computing device E estimates a pointing position according to the movement and posture change of the remote controller 200 measured by the remote controller 200, and moves and displays the position of the cursor, which is an object displayed on the television screen. Further, the calculation mechanism E calibrates the reference pose of the remote controller 200 which is the basis of the estimation of the pointing position according to the image picked up by the camera 11.
  • the remote controller 200 includes a posture change amount measuring unit 30 for measuring a posture change amount therein, and a marker portion M which becomes a marker when the remote control 200 is captured by the camera 11 at a tip end thereof. ).
  • the posture change amount measuring unit 30 includes a geomagnetic sensor 31, an acceleration sensor 32 that is a motion sensor, and an angular velocity sensor 33.
  • the remote controller 200 has a roughly rectangular parallelepiped shape, and operation buttons used for channel selection and the like are arranged on one side thereof, and the direction of the front end face is indicated by each sensor as an instruction direction PL. To be detected.
  • the geomagnetic sensor 31 is for detecting a direction indicated by the remote controller 200 in a horizontal plane according to the geomagnetic.
  • the direction indicated by the geomagnetic sensor 31 is obtained as a reference for detecting rotation in the horizontal plane.
  • the attitude change when the user changes the instruction direction PL of the remote controller 200 is measured not by the geomagnetic sensor 31 but by the motion sensor having excellent time resolution.
  • the acceleration sensor 32 is for measuring the acceleration when the remote control 200 is operated by the user. This acceleration is used to compensate for the angular velocity measured by the angular velocity sensor 33.
  • the angular velocity sensor 33 has a central axis passing through both end faces of the remote controller 200 as a roll axis, an axis perpendicular to the opposite side surfaces on which the operation buttons are not arranged, and a pitch axis (horizontal axis), and perpendicular to a side at which the operation buttons are arranged.
  • one axis is defined as the yaw axis (vertical axis)
  • the angular velocity (pitching) in the pitch axis direction and the angular velocity in the yawing axis direction are measured. That is, this angular velocity sensor 33 is provided with two detection elements so that the angular velocity of a pitch axis and a yaw axis direction can be detected.
  • the marker unit M is four light sources housed in an internal space having a substantially rectangular parallelepiped shape at the distal end of the remote controller 200, and each of the LEDs 1 having different emission colors, and the same number of openings as the plurality of LEDs 1 Light-shielding body 2 having 21).
  • the four LEDs 1 are arranged to form a substantially square shape when viewed from a direction perpendicular to the front end surface of the remote controller 200.
  • the four-color LED 1 is used in this embodiment, all of the same color may be used.
  • the light shielding body 2 is spaced apart by a predetermined distance to the light emitting side of the plurality of LEDs 1, and each opening portion 21 is disposed on the optical axis of each LED 1. That is, each opening part 21 is also arrange
  • FIG. As will be described later, when the marker mechanism M faces the television screen in front, the light from the four LEDs 1 is picked up by the camera 11 so that the remote control 200 is predetermined with respect to the television screen. In the case of having a posture inclined at an angle, the number and positions of the LEDs 1 captured by the camera 11 change according to the inclination direction.
  • the camera 11 is arranged at the center of the upper bezel of the television 100, and is configured to capture an image in a direction perpendicular to the television screen.
  • the camera 11 has a predetermined imaging range at a three-dimensional angle, and only light incident to the camera 11 among the light traveling in a direction perpendicular to the television screen is imaged brightly.
  • the computing device E is a so-called computer provided with a CPU, a memory, an input / output device, an A / D, a D / A converter, and the like.
  • the program for the image display system 300 which is stored in the memory in cooperation with various devices, is used. It is configured to perform the functions as the pointing angle calculator 15, the position calculator 16, the remote control attitude estimation unit 6, and the calibration unit 7 for execution.
  • the pointing position estimator 4 is a reference posture of the remote controller 200, and an attitude change amount of the remote controller 200 measured by the geomagnetic sensor 31, the acceleration sensor 32, and the angular velocity sensor 33. In accordance with the above, the pointing position (coordinate) on the television screen is calculated.
  • the pointing position estimating unit 4 sets the direction of instruction PL when the remote controller 200 is in the reference position as the reference direction BPL, and the current direction of direction PL with respect to the reference direction BPL.
  • a pointing angle calculation unit 15 for calculating a pointing angle ⁇ , which is a pitch angle and yaw angle formed by a), and a current pointing direction PL according to the pointing angle ⁇ calculated by the pointing angle calculation unit 41.
  • a coordinate calculating section 42 for calculating a pointing point which is a point at which the television screen intersects.
  • the remote controller 200 includes an operation angle range CR for receiving an angle change as an operation input by a solid angle centered on the reference indicating direction BPL in the reference posture. Is set in advance. For example, when the remote controller 200 faces the television screen in front, and the reference indicating direction BPL points to the center of the television screen, the yaw angle or pitch angle is changed from the reference posture to change the pointing direction PL.
  • the operating angle range CR is set based on the angle at which the points indicated by) become the top, bottom, left and right vicinity of the bezel.
  • FIG. 29A is an example, and the reference direction adjustment part 7 mentioned later is made to calibrate into nine types of reference attitudes according to the instruction direction PL of the said remote control 200.
  • FIG. 29A is an example, and the reference direction adjustment part 7 mentioned later is made to calibrate into nine types of reference attitudes according to the instruction direction PL of the said remote control 200.
  • this pointing angle calculation part 15 has the current instruction direction PL in the reference
  • the position calculator 16 indicates that the current pointing direction PL is estimated to point on the television screen according to the reference point estimated to be indicated by the reference pointing direction BPL on the television screen and the current pointing angle ⁇ . Calculate the coordinates of a point.
  • the display unit 18 shown in FIG. 28 causes the cursor to move and display by the coordinates on the television screen of the pointing position calculated by the position calculator 42.
  • the image capturing posture estimating section 6 estimates the image capturing posture, which is a posture at the time of imaging the object, according to the number or position of the LEDs 1 in the image picked up by the camera 11. More specifically, the remote control attitude estimation section 6 determines which of the nine square areas A provided in the center of the television screen, the instruction direction PL of the remote control 200 instructs, and the area thereof. Estimate the posture needed to indicate (A). As shown in Fig. 27 and the like, the nine areas A are arranged in three vertically, three horizontally, and square sides. In addition, the number given about each area A of FIG. 27 is corresponded with the number shown in the following description and drawing.
  • the number and positions of the LEDs 1 appearing in the image picked up by the camera 11 in a state in which the instruction direction PL of the remote controller 200 indicates each area A are shown in FIG. 30. Each one is unique. The inherent relationship between the indicated area A and the number and position of the LEDs 1 in the image may be caused by the configuration of the marker portion M, but the details will be described later. Shall be.
  • the correspondence storage unit stores the correspondence between the number and position of the LEDs 1 in the image and the posture at the time of image capture. 61 and a posture output section 62 for outputting a posture at the time of imaging corresponding to the number and position of the LEDs 1 among the images picked up by the camera 11 with reference to the correspondence relationship storage section 61. Consists of
  • the correspondence relationship storage unit 61 stores the area A indicated by the remote controller 200 shown in FIG. 30 and the image picked up when the area A is indicated as reference images. In addition, when each reference image is captured, the attitude of the remote controller 200 is also stored in association with each image. For example, when the remote control 200 points to the fifth area A of the television screen center part, since the remote control 200 faces the television screen in front, it is associated with the image when the fifth area A is pointing. , Pitch angle and yaw angle are stored as zero.
  • the correspondence relationship storage section 61 stores pitches as predetermined values and yaw angles as zero in association with an image when the remote control 200 points the second and eighth areas A to 4, and 4
  • the reference image in the case where the remote control 200 points to the number 6 and area A, the pitch angle is zero and the yaw angle is stored as a predetermined value.
  • the intrinsic pitch angle and yaw angle are also stored in association with the reference image of the first, third, seventh and ninth diagonal areas A as the posture of the remote controller 200, respectively.
  • the posture output section 62 acquires from the correspondence relationship storing section 61 that the number of the LEDs 1 in the captured image, the position and the pattern in the image match, and captures the posture associated with the image. It outputs as a posi- tion posture. For example, when the reference image stored in the correspondence relationship storage section 61 and the image currently being captured are not perfectly matched, the posture associated with the nearest reference image may be used as the posture at the time of imaging. The posture associated with the position of the LED 1 in the reference image and the position of the LED 1 currently being captured may be corrected to form an image at the time of imaging.
  • the calibration section 7 is configured to calibrate the reference posture into a posture at the time of imaging estimated by the posture estimating section 6.
  • the calibration by this calibration part 7 is performed every predetermined time.
  • the reason why the number and positions of the LEDs 1 in the image picked up in the state where each area A is instructed by using the marker portion M of the present embodiment will be described.
  • the photograph at the time of imaging in the state which actually indicated each area A in the image display system 100 of this embodiment is shown in FIG.
  • the LED 1 disposed in the upper left portion is the LED 1 (green) and the upper right portion.
  • the LED 1 arranged in the LED (white), the LED 1 arranged in the lower left part is called LED (red), and the LED 1 arranged in the lower right part is also called LED (blue).
  • the remote control 200 indicates the second area A by changing the pitch angle upward from the state of the fifth area A
  • the remote controller 200 since the remote controller 200 is inclined, the light emitted from the LED (green) in the upper left of the four LEDs 1 is the lower left opening 21 on the optical axis of the LED (white) in the lower left. 2 is reached through the area A, and the light emitted from the LED (white) in the upper right portion passes through the opening 21 in the lower right portion on the optical axis of the LED (blue) in the lower right portion. Reach the area A of the bun.
  • the LEDs (red) and the LEDs (blue) in the lower left and the lower right do not reach the second area A through any of the openings 21, and both are shielded by the light shielding body 2. .
  • only two LEDs 1 are captured in the lower side of the image.
  • the phenomenon that occurs when pointing to the second area A is similarly established for the fourth, sixth and eighth areas A in the up, down, left, and right directions with respect to the fifth area A, and two LEDs respectively.
  • Only (1) is common in the point of imaging. More specifically, two LEDs 1 are imaged at positions of the lower half of the image for No. 2, the right half of the image for No. 4, the left half of the image for No. 6, and the upper half of the image for No. 8, In the case where the remote control 200 points the area A in the up, down, left, and right directions with respect to the five areas A, it can be seen that each can be individually determined.
  • the pitch angle is changed upward from the state in which the remote control 200 points to the fifth area A, and the yaw angle is changed in the counterclockwise direction so that the third area A is indicated.
  • the yaw angle is changed in the counterclockwise direction so that the third area A is indicated.
  • FIG. 32 three areas are opened through the opening 21 in which the LED (back) in the upper right part of the four LEDs 1 is opened on the optical axis of the LED (red) on its diagonal. (A) can be reached.
  • the three other LEDs 1 do not reach the third area A even through any of the openings 21, and are shielded by the light shielding body 2.
  • only one LED 1 is imaged in the lower left of the image.
  • the phenomenon that occurs when the third area A is indicated is similarly established for the first, seventh and ninth areas A, and is common in that only one LED 1 is picked up. More specifically, since one LED 1 is imaged in the lower right part of an image about No. 1, the lower left part about No. 3, the upper right part about No. 7, and the upper left part about No. 9, each area A When indicated by the remote control 200, each can be determined individually.
  • the remote controller attitude estimation unit 6 may estimate the remote controller attitude at the time of imaging.
  • the absolute posture of the current remote controller 200 can be estimated from the pattern determination according to the image, the basis for estimating the pointing position according to the estimated image at the time of image capturing even if the user does not move the remote controller 200 in the designated posture. You can calibrate the reference pose to be.
  • the image display system 300 of this embodiment since a calibration start operation by a user is not required like conventionally, it is possible to automatically calibrate a reference posture regularly. As a result, a large gap between the point indicated by the instruction direction PL of the remote controller 200 and the estimated pointing point can be prevented while the user is using it, so that each point can be kept almost always in a consistent state.
  • position at the time of imaging of the remote controller 200 is estimated by the pattern matching according to the number and position of LED1 in the above-mentioned image, it is possible to raise the estimation degree of attitude
  • a pointing device using a television and a remote controller has been described, but the present invention can be similarly applied to other relative pointing methods using a projector, a PC, or the like.
  • the light source of a marker mechanism is not limited to four, A plurality may be sufficient. That is, the number and position of the light sources in the image picked up by the camera may be set so as to have a unique relationship in accordance with the direction in which the object is instructed.
  • each of the calibration modes is calibrated in different reference postures according to the area currently instructed by the remote control.
  • the control range may be appropriately changed in accordance with the area currently instructed by the remote control.
  • the image display apparatus 100 estimates a posture of the remote controller during imaging corresponding to the direction of the imaging of the remote controller 200 according to the number or positions of the light sources of the remote controller 200 that have been picked up.
  • the image display apparatus 100 calibrates the reference attitude of the remote controller corresponding to the reference direction of the remote controller 200 to the attitude of the remote controller during the estimated imaging.
  • the image display apparatus 100 estimates the pointing position according to the calibrated reference attitude and the posture change amount measured by the motion sensor installed in the remote controller.
  • the image display apparatus 100 displays an object at the pointing position on the pointing object according to the estimated pointing position.
  • the posture at the time of imaging of the workpiece is estimated according to the number and position of light sources in the image captured by the camera 11, and the posture at the time of imaging.
  • calibrating the reference posture which is the basis of the estimation of the pointing position, it is possible to automate all calibration operations. In other words, it is possible to eliminate the need for the user to maintain the workpiece in a reference position before calibration or to perform a special operation as a trigger for appropriately initiating calibration.
  • the reference posture can be calibrated periodically, the relative pointing method can be matched again before a large deviation occurs between the position indicated by the direction of the workpiece and the estimated pointing position. Practically, each point can be consistent.
  • the pointing device is configured between the television and the remote control, but for example, the pointing device may be configured between the other display and the remote control.
  • the display may be configured by a projector or the like, and a pointer may be displayed on the projection screen.
  • the remote control direction detecting mechanism (the mechanism comprising the camera, the first remote control detecting unit and the second remote control detecting unit in the fourth embodiment) may be other than using a camera and an LED marker.
  • a sound wave or radio wave generation source may be provided at both ends of the display, and the triangulation technique may detect the relative position of the remote control with respect to the display, and may calculate the remote control direction angle.
  • it may be a method using ultrasonic waves.
  • An ultrasonic transmitter is provided in the remote control, and two ultrasonic receivers are provided at different positions of the television.
  • the orientation of the remote control seen from the television can be measured from the triangulation principle.
  • three or more ultrasonic wave receivers may be used, and when three or more ultrasonic wave receivers are used, more accurate measurement is possible.
  • a radio wave may be used.
  • the remote control is provided with a radio transmitter and two radio receivers at different positions of the television. Can be measured.
  • three or more radio wave receivers may be used, and when three or more radio receivers are used, measurement of higher accuracy is possible.
  • the reference angle detection mechanism is not limited to the one using an orientation sensor.
  • three-dimensional positions of the front and rear ends of the remote controller are detected using radio waves or sound waves, and the direction of the instruction is directly measured from these two coordinates. You can do it.
  • the reference direction is directed toward the screen center by replacing the currently used pointing angle with the reference pointing angle, but the reference direction may be adjusted by other calculation methods such as coordinate changes. In short, the reference direction may be adjusted based on the reference direction and the reference angle.
  • center of the screen is set as the reference point in the above embodiment, for example, a point other than the center of the screen on the display may be set as the reference point.
  • the present invention relates to an image display apparatus which controls a pointer to be displayed on a display, and is generally applicable to a television system.

Landscapes

  • Engineering & Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Position Input By Displaying (AREA)

Abstract

L'invention concerne un appareil d'affichage d'image qui se règle automatiquement de telle sorte que la direction d'indication d'une télécommande et la position d'un pointeur sur un écran correspondent approximativement l'une à l'autre, ainsi que son procédé de pointage. L'appareil d'affichage d'image est pourvu : d'une unité de détection d'angle de référence (4) pour détecter un angle de référence (AC), qui est un angle formé à partir de la direction d'indication d'une télécommande (200) et de la direction de référence (CO) établie par rapport à un dispositif d'affichage (100) ; d'une unité de détection de direction de télécommande (3) pour détecter une direction de télécommande (RD), qui est une direction faisant face à la télécommande (200) à partir d'un point de référence (PO) qui est établi sur le dispositif d'affichage (100) ; d'une unité de réglage de direction standard (14) pour régler une direction standard (SD), sur la base de l'angle de référence (AC) et de la direction de télécommande (SD), de façon à avoir le point de référence (PO) sur le dispositif d'affichage (100) lorsqu'il fait face à la direction standard (SD) de la télécommande (200).
PCT/KR2014/009273 2013-10-02 2014-10-01 Appareil d'affichage d'image et son procédé de pointage WO2015050380A1 (fr)

Priority Applications (2)

Application Number Priority Date Filing Date Title
EP14850816.1A EP3054693B1 (fr) 2013-10-02 2014-10-01 Appareil d'affichage d'image et son procédé de pointage
US15/027,108 US9910507B2 (en) 2013-10-02 2014-10-01 Image display apparatus and pointing method for same

Applications Claiming Priority (10)

Application Number Priority Date Filing Date Title
JP2013-207599 2013-10-02
JP2013207599A JP6200756B2 (ja) 2013-10-02 2013-10-02 ポインティング装置、ポインティング方法、及び、ポインティング装置用プログラム
JP2013-222280 2013-10-25
JP2013222280 2013-10-25
JP2013-242589 2013-11-25
JP2013242589 2013-11-25
JP2014173247A JP6370165B2 (ja) 2013-10-25 2014-08-27 ポインティング装置、ポインティング方法、プログラム及び画像表示装置
JP2014-173247 2014-08-27
KR1020140132316A KR102207256B1 (ko) 2013-10-02 2014-10-01 화상 표시 장치 및 그의 포인팅 방법
KR10-2014-0132316 2014-10-01

Publications (1)

Publication Number Publication Date
WO2015050380A1 true WO2015050380A1 (fr) 2015-04-09

Family

ID=52778923

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2014/009273 WO2015050380A1 (fr) 2013-10-02 2014-10-01 Appareil d'affichage d'image et son procédé de pointage

Country Status (1)

Country Link
WO (1) WO2015050380A1 (fr)

Cited By (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN106843266A (zh) * 2016-12-30 2017-06-13 歌尔科技有限公司 一种飞行器的遥控器、朝向控制系统和方位指示方法
CN108369477A (zh) * 2015-12-22 2018-08-03 索尼公司 信息处理装置、信息处理方法和程序

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002062981A (ja) 2000-08-16 2002-02-28 Nippon Hoso Kyokai <Nhk> 表示画面指示装置
JP2006155345A (ja) 2004-11-30 2006-06-15 Sharp Corp 遠隔制御装置及び表示装置
JP2008181198A (ja) 2007-01-23 2008-08-07 Funai Electric Co Ltd 画像表示システム
JP2008227605A (ja) 2007-03-08 2008-09-25 Hitachi Ltd Ｌｅｄポインティングリモコンとこれを用いた映像表示装置、及び映像システム
JP2011170401A (ja) * 2010-02-16 2011-09-01 Taito Corp ポインティングデバイス等を用いた入力選択装置及び当該入力選択装置を構築するためのコンピュータープログラム
KR20110099461A (ko) * 2010-03-02 2011-09-08 (주)실리콘화일 포인팅 제어 장치 및 제어 방법
KR20120037278A (ko) * 2010-10-11 2012-04-19 연세대학교 산학협력단 3차원 디스플레이 시스템
KR20120055434A (ko) * 2010-11-22 2012-05-31 삼성전자주식회사 디스플레이 시스템 및 그 디스플레이 방법
KR20120126508A (ko) * 2011-05-12 2012-11-21 김석중 포인터를 사용하지 않는 가상 터치 장치에서의 터치 인식 방법

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2002062981A (ja) 2000-08-16 2002-02-28 Nippon Hoso Kyokai <Nhk> 表示画面指示装置
JP2006155345A (ja) 2004-11-30 2006-06-15 Sharp Corp 遠隔制御装置及び表示装置
JP2008181198A (ja) 2007-01-23 2008-08-07 Funai Electric Co Ltd 画像表示システム
JP2008227605A (ja) 2007-03-08 2008-09-25 Hitachi Ltd Ｌｅｄポインティングリモコンとこれを用いた映像表示装置、及び映像システム
JP2011170401A (ja) * 2010-02-16 2011-09-01 Taito Corp ポインティングデバイス等を用いた入力選択装置及び当該入力選択装置を構築するためのコンピュータープログラム
KR20110099461A (ko) * 2010-03-02 2011-09-08 (주)실리콘화일 포인팅 제어 장치 및 제어 방법
KR20120037278A (ko) * 2010-10-11 2012-04-19 연세대학교 산학협력단 3차원 디스플레이 시스템
KR20120055434A (ko) * 2010-11-22 2012-05-31 삼성전자주식회사 디스플레이 시스템 및 그 디스플레이 방법
KR20120126508A (ko) * 2011-05-12 2012-11-21 김석중 포인터를 사용하지 않는 가상 터치 장치에서의 터치 인식 방법

Cited By (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN108369477A (zh) * 2015-12-22 2018-08-03 索尼公司 信息处理装置、信息处理方法和程序
EP3396509A4 (fr) * 2015-12-22 2018-12-05 Sony Corporation Dispositif de traitement d'informations, procédé de traitement d'informations et programme
CN108369477B (zh) * 2015-12-22 2021-10-29 索尼公司 信息处理装置、信息处理方法和程序
CN106843266A (zh) * 2016-12-30 2017-06-13 歌尔科技有限公司 一种飞行器的遥控器、朝向控制系统和方位指示方法
CN106843266B (zh) * 2016-12-30 2024-04-05 歌尔科技有限公司 一种飞行器的遥控器、朝向控制系统和方位指示方法

Similar Documents

Publication Publication Date Title
WO2016065626A1 (fr) Procédé et appareil de traitement de fuite de gaz, et véhicule aérien
WO2017078356A1 (fr) Dispositif d&#39;affichage et procédé d&#39;affichage d&#39;image associé
WO2015152692A1 (fr) Appareil et procédé de génération d&#39;image périphérique de véhicule
WO2019231042A1 (fr) Dispositif d&#39;authentification biométrique
EP3571832A1 (fr) Module d&#39;appareil photo et dispositif d&#39;appareil photo
WO2011074810A2 (fr) Appareil de numérisation pratique et procédé de commande de celui-ci
WO2015141925A1 (fr) Appareil photographique, son procédé de commande, et support d&#39;enregistrement lisible par ordinateur
WO2013147464A1 (fr) Dispositif à écran tactile multipoint
WO2016175424A1 (fr) Terminal mobile, et procédé de commande associé
WO2019135475A1 (fr) Appareil électronique et son procédé de commande
WO2022050507A1 (fr) Procédé et système de surveillance d&#39;un module de génération d&#39;énergie photovoltaïque
WO2018212608A1 (fr) Système de marquage mobile, procédé de commande de dispositif de marquage mobile, et support d&#39;enregistrement lisible par ordinateur
WO2020101420A1 (fr) Procédé et appareil de mesurer des caractéristiques optiques d&#39;un dispositif de réalité augmentée
WO2017090833A1 (fr) Dispositif de prise de vues, et procédé de commande associé
WO2019208915A1 (fr) Dispositif électronique pour acquérir une image au moyen d&#39;une pluralité de caméras par ajustage de la position d&#39;un dispositif extérieur, et procédé associé
WO2019143050A1 (fr) Dispositif électronique et procédé de commande de mise au point automatique de caméra
WO2019125036A1 (fr) Procédé de traitement d&#39;image et appareil d&#39;affichage associé
WO2020153703A1 (fr) Appareil de caméra et son procédé de mise au point automatique
WO2015050380A1 (fr) Appareil d&#39;affichage d&#39;image et son procédé de pointage
WO2021145667A1 (fr) Procédé et dispositif de commande de mouvement de caméra
WO2021246758A1 (fr) Dispositif électronique et son procédé de fonctionnement
WO2016099165A1 (fr) Capteur d&#39;image, dispositif d&#39;acquisition d&#39;image le comprenant, et terminal portable comprenant le dispositif
WO2021210875A1 (fr) Dispositif électronique permettant de détecter un défaut dans une image sur la base d&#39;une différence entre des sous-images acquises par un capteur à photodiodes multiples, et son procédé de fonctionnement
WO2022260273A1 (fr) Appareil électronique et son procédé de commande
WO2022097930A1 (fr) Dispositif électronique et procédé d&#39;affichage pour celui-ci

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14850816

Country of ref document: EP

Kind code of ref document: A1

REEP Request for entry into the european phase

Ref document number: 2014850816

Country of ref document: EP

WWE Wipo information: entry into national phase

Ref document number: 2014850816

Country of ref document: EP

NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 15027108

Country of ref document: US