WO2023157338A1 - 情報処理装置およびデバイス位置推定方法 - Google Patents

情報処理装置およびデバイス位置推定方法 Download PDF

Info

Publication number
WO2023157338A1
WO2023157338A1 PCT/JP2022/030070 JP2022030070W WO2023157338A1 WO 2023157338 A1 WO2023157338 A1 WO 2023157338A1 JP 2022030070 W JP2022030070 W JP 2022030070W WO 2023157338 A1 WO2023157338 A1 WO 2023157338A1
Authority
WO
WIPO (PCT)
Prior art keywords
estimation
processing unit
unit
sensor data
estimated
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/030070
Other languages
English (en)
French (fr)
Japanese (ja)
Inventor
憲三 西川
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Sony Interactive Entertainment Inc
Original Assignee
Sony Interactive Entertainment Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Sony Interactive Entertainment Inc filed Critical Sony Interactive Entertainment Inc
Priority to JP2024500932A priority Critical patent/JPWO2023157338A1/ja
Priority to US18/726,134 priority patent/US12314486B1/en
Publication of WO2023157338A1 publication Critical patent/WO2023157338A1/ja
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • G—PHYSICS
    • G01—MEASURING; TESTING
    • G01P—MEASURING LINEAR OR ANGULAR SPEED, ACCELERATION, DECELERATION, OR SHOCK; INDICATING PRESENCE, ABSENCE, OR DIRECTION, OF MOVEMENT
    • G01P15/00—Measuring acceleration; Measuring deceleration; Measuring shock, i.e. sudden change of acceleration
    • G01P15/18—Measuring acceleration; Measuring deceleration; Measuring shock, i.e. sudden change of acceleration in two or more dimensions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/02—Input arrangements using manually operated switches, e.g. using keyboards or dials
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G06T7/73—Determining position or orientation of objects or cameras using feature-based methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/041—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means
    • G06F3/044—Digitisers, e.g. for touch screens or touch pads, characterised by the transducing means by capacitive means
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30244—Camera pose

Definitions

  • the present disclosure relates to technology for estimating the location of a device.
  • Patent Document 1 discloses an information processing apparatus that identifies representative coordinates of marker images from an image of a device having a plurality of markers and derives position information and orientation information of the device using the representative coordinates of the marker images. .
  • An information processing apparatus disclosed in Patent Document 1 identifies a first bounding box surrounding an area in which pixels having a first luminance or more are continuous in a captured image, and specifies a second bounding box having a second luminance higher than the first luminance within the first bounding box.
  • a second bounding box surrounding an area in which pixels having brightness equal to or higher than the brightness are continuous is specified, and representative coordinates of the marker image are derived based on the pixels inside the first bounding box or inside the second bounding box.
  • Patent Document 2 discloses an input device provided with a plurality of light emitting parts and a plurality of operating members.
  • the light-emitting unit of the input device is photographed by a camera provided in the head-mounted device, and the position and orientation of the input device are calculated based on the detected position of the light-emitting unit.
  • An object of the present disclosure is to provide a technique for estimating a device position.
  • the device may be an input device having an operation member, but may also be a device that does not have an operation member and is simply a tracking target.
  • an information processing apparatus that estimates a position of a device, and includes a captured image acquisition unit that acquires a captured image of the device, acceleration of the device, and / Or a sensor data acquisition unit that acquires sensor data indicating angular velocity, an estimation processing unit that estimates the position of the device based on an image of the device, and a contact determination that determines whether the user is touching the device and a stationary determination unit that determines whether the device is stationary based on the sensor data.
  • the estimation processing unit fixes the estimated position of the device.
  • An information processing apparatus that estimates the position of a device, and includes a captured image acquisition unit that acquires a captured image of the device, and sensor data that indicates acceleration and/or angular velocity of the device. a sensor data acquisition unit that acquires , a contact determination unit that determines whether the user is touching the device, a stationary determination unit that determines whether the device is stationary based on the sensor data, and a device and an estimation processing unit that estimates the position.
  • the estimation processing unit includes a first estimation processing unit that estimates the position of the device based on the captured image of the device, a second estimation processing unit that estimates the position of the device based on the sensor data, and a first estimation processing unit.
  • a third estimation processing unit that derives the device location based on the device location estimated by the unit and the device location estimated by the second estimation processing unit.
  • the estimation processing unit fixes the estimated position of the device.
  • a device position estimation method includes the steps of obtaining an image of a device, obtaining sensor data indicating the acceleration and/or angular velocity of the device, and based on the image of the device being photographed, , estimating the position of the device, determining whether the user is touching the device, determining whether the device is stationary based on the sensor data, and determining whether the user is touching the device. and fixing the estimated device position if it is determined that the device is stationary and the device is stationary.
  • a device position estimation method includes the steps of acquiring an image captured by an imaging device, acquiring sensor data indicating the acceleration and/or angular velocity of the device, and determining whether the user is touching the device. a step of determining whether the device is stationary based on sensor data; and an estimating step of estimating the position of the device, wherein the estimating step is an image of the device a first estimating step of estimating the position of the device based on the sensor data; a second estimating step of estimating the position of the device based on the sensor data; the position of the device estimated in the first estimating step; a third estimation step of estimating the location of the device based on the location of the device estimated in , wherein when it is determined that the user has not touched the device and the device is stationary, the estimation step includes , to fix the estimated device location.
  • FIG. 2 is a diagram showing functional blocks of an input device;
  • FIG. 9 is a flowchart showing position and orientation estimation processing; It is a figure which shows the internal structure of an estimation process part. 4 is a flowchart showing position fixing processing;
  • FIG. 1 shows a configuration example of an information processing system 1 in an embodiment.
  • An information processing system 1 includes an information processing device 10, a recording device 11, a head-mounted display (HMD) 100, an input device 16 held by a user and operated with fingers, and an output device 15 for outputting images and sounds.
  • Output device 15 may be a television.
  • the information processing device 10 is connected to an external network 2 such as the Internet via an access point (AP) 17 .
  • the AP 17 has the functions of a wireless access point and router, and the information processing device 10 may be connected to the AP 17 by a cable or by a known wireless communication protocol.
  • the recording device 11 records applications such as system software and game software.
  • the information processing device 10 may download game software from the content server to the recording device 11 via the network 2 .
  • the information processing device 10 executes game software and supplies game image data and audio data to the HMD 100 .
  • the information processing device 10 and the HMD 100 may be connected by a known wireless communication protocol, or may be connected by a cable.
  • the HMD 100 is a display device that displays an image on a display panel positioned in front of the user's eyes when worn by the user.
  • the HMD 100 separately displays a left-eye image on the left-eye display panel and a right-eye image on the right-eye display panel. These images constitute parallax images viewed from left and right viewpoints, and achieve stereoscopic vision. Since the user views the display panel through an optical lens, the information processing apparatus 10 supplies the HMD 100 with parallax image data corrected for optical distortion caused by the lens.
  • a user wearing the HMD 100 does not need the output device 15, but by preparing the output device 15, another user can view the display image of the output device 15.
  • the information processing apparatus 10 may cause the output device 15 to display the same image as the image viewed by the user wearing the HMD 100, or may cause the output device 15 to display a different image. For example, when a user wearing an HMD and another user play a game together, the output device 15 may display a game image from the viewpoint of the character of the other user.
  • the information processing apparatus 10 and the input device 16 may be connected by a known wireless communication protocol, or may be connected by a cable.
  • the input device 16 has a plurality of operation members such as operation buttons, and the user operates the operation members with fingers while holding the input device 16 .
  • the input device 16 is used as a game controller.
  • the input device 16 includes an inertial measurement unit (IMU) including a 3-axis acceleration sensor and a 3-axis angular velocity sensor, and transmits sensor data to the information processing apparatus 10 at a predetermined cycle (800 Hz, for example).
  • IMU inertial measurement unit
  • the game of the embodiment handles not only the operation information of the operation members of the input device 16, but also the speed, position, posture, etc. of the input device 16 as operation information, and reflects them on the movement of the player character in the virtual three-dimensional space.
  • the operation information of the operation member may be used as information for moving the player character
  • the operation information such as the speed, position and attitude of the input device 16 may be used as information for moving the arm of the player character.
  • the movement of the input device 16 is reflected in the movement of the player character with the weapon, thereby realizing intuitive operations for the user and increasing the sense of immersion in the game.
  • the input device 16 In order to track the position and orientation of the input device 16, the input device 16 is provided with a plurality of markers (light emitting units) that can be photographed by the imaging device 14.
  • the information processing apparatus 10 has a function of analyzing a photographed image of the input device 16 and estimating the position and orientation of the input device 16 in real space (hereinafter also referred to as a "first estimation function").
  • a plurality of imaging devices 14 are mounted on the HMD 100 .
  • the plurality of imaging devices 14 are attached at different positions on the front surface of the HMD 100 in different postures so that the total imaging range, which is the sum of the respective imaging ranges, includes the entire field of view of the user.
  • the imaging device 14 comprises an image sensor capable of acquiring images of the markers of the input device 16 .
  • the imaging device 14 has a visible light sensor such as a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor used in general digital video cameras.
  • the marker emits non-visible light imaging device 14 has a non-visible light sensor.
  • a plurality of imaging devices 14 capture images in front of the user at a predetermined cycle (for example, 120 frames/second) at synchronized timing, and transmit image data of the captured real space to the information processing device 10 .
  • the information processing device 10 executes the first estimation function to specify the positions of the multiple marker images of the input device 16 included in the captured image. Although one input device 16 may be photographed by a plurality of imaging devices 14 at the same timing, the mounting positions and mounting orientations of the imaging devices 14 are known, so the information processing device 10 synthesizes a plurality of captured images. to identify the position of the marker image.
  • the three-dimensional shape of the input device 16 and the position coordinates of the plurality of markers arranged on its surface are known, and the information processing apparatus 10 calculates the input device 16 based on the position coordinates of the plurality of marker images in the captured image. Estimate the position and pose of the in real space.
  • the position of the input device 16 is estimated as world coordinate values in a three-dimensional space with the reference position as the origin, and the reference position may be the position coordinates (latitude, longitude, altitude (elevation)) set before the start of the game.
  • the information processing apparatus 10 of the embodiment has a function of analyzing sensor data transmitted from the input device 16 and estimating the velocity, position and orientation of the input device 16 in real space (hereinafter also referred to as a "second estimation function"). ).
  • the information processing apparatus 10 derives the position and orientation of the input device 16 using the estimation result of the first estimation function and the estimation result of the second estimation function.
  • the information processing apparatus 10 of the embodiment integrates the estimation result by the first estimation function and the estimation result by the second estimation function using the state estimation technique using the Kalman filter, so that the state of the input device 16 at the current time is Estimate the state with high accuracy.
  • FIG. 2 shows an example of the external shape of the HMD 100.
  • the HMD 100 is composed of an output mechanism section 102 and a mounting mechanism section 104 .
  • the mounting mechanism unit 104 includes a mounting band 106 that is worn by the user and wraps around the head to fix the HMD 100 to the head.
  • the mounting band 106 has a material or structure whose length can be adjusted according to the circumference of the user's head.
  • the output mechanism unit 102 includes a housing 108 shaped to cover the left and right eyes when the user wears the HMD 100, and has a display panel inside that faces the eyes when the HMD 100 is worn.
  • the display panel may be a liquid crystal panel, an organic EL panel, or the like.
  • the housing 108 further includes a pair of left and right optical lenses positioned between the display panel and the user's eyes to expand the viewing angle of the user.
  • the HMD 100 may further include speakers and earphones at positions corresponding to the ears of the user, and may be configured to connect external headphones.
  • a plurality of imaging devices 14a, 14b, 14c, and 14d are provided on the front outer surface of the housing 108. Based on the front direction of the user's face, the imaging device 14a is attached to the upper right corner of the front outer surface so that the camera optical axis faces diagonally upward to the right, and the imaging device 14b is installed so that the camera optical axis faces diagonally upward to the left.
  • the imaging device 14c is attached to the lower right corner of the front outer surface so that the camera optical axis is directed diagonally downward to the right, and the imaging device 14d is installed so that the camera optical axis is diagonally downward to the left. It is mounted in the lower left corner of the front side outer surface facing up.
  • the HMD 100 transmits sensor data detected by an inertial measurement unit (IMU) and image data captured by an imaging device 14 to the information processing device 10, and receives game image data and game audio data generated by the information processing device 10. do.
  • IMU inertial measurement unit
  • FIG. 3 shows the functional blocks of the HMD 100.
  • the control unit 120 is a main processor that processes and outputs various data such as image data, audio data, sensor data, and commands.
  • Storage unit 122 temporarily stores data, instructions, and the like processed by control unit 120 .
  • the IMU 124 acquires sensor data regarding the motion of the HMD 100 .
  • the IMU 124 may include at least a triaxial acceleration sensor and a triaxial angular rate sensor.
  • the IMU 124 detects the value (sensor data) of each axial component at a predetermined cycle (800 Hz, for example).
  • the communication control unit 128 transmits data output from the control unit 120 to the external information processing device 10 by wired or wireless communication via a network adapter or an antenna.
  • the communication control unit 128 also receives data from the information processing device 10 and outputs the data to the control unit 120 .
  • control unit 120 When the control unit 120 receives game image data and game sound data from the information processing device 10, the control unit 120 supplies the data to the display panel 130 for display, and also supplies the data to the sound output unit 132 for sound output.
  • the display panel 130 includes a left-eye display panel 130a and a right-eye display panel 130b, and each display panel displays a pair of parallax images.
  • the control unit 120 also causes the communication control unit 128 to transmit the sensor data from the IMU 124 , the audio data from the microphone 126 , and the captured image data from the imaging device 14 to the information processing device 10 .
  • FIG. 4(a) shows the shape of the input device 16a for the left hand.
  • the input device 16a for the left hand includes a case body 20, a plurality of operation members 22a, 22b, 22c, and 22d operated by a user (hereinafter referred to as "operation members 22" unless otherwise specified), and a case body 20. and a plurality of markers 30 for emitting light to the outside.
  • the marker 30 may have an exit portion with a circular cross section.
  • the operation member 22 may include an analog stick for tilting operation, a push button, or the like.
  • the case body 20 has a grip portion 21 and a curved portion 23 that connects the top portion of the case body and the bottom portion of the case body. The user operates the operation members 22a, 22b, 22c, and 22d with the thumb of the left hand while gripping the grip portion 21. As shown in FIG.
  • FIG. 4(b) shows the shape of the input device 16b for the right hand.
  • the input device 16b for the right hand includes a case body 20, a plurality of operation members 22e, 22f, 22g, and 22h operated by the user (hereinafter referred to as "operation members 22" unless otherwise specified), and the case body 20. and a plurality of markers 30 for emitting light to the outside.
  • the operation member 22 may include an analog stick for tilting operation, a push button, or the like.
  • the case body 20 has a grip portion 21 and a curved portion 23 that connects the top portion of the case body and the bottom portion of the case body. The user operates the operation members 22e, 22f, 22g, and 22h with the thumb of the right hand while gripping the grip portion 21. As shown in FIG.
  • FIG. 5 shows the shape of the input device 16b for the right hand.
  • the input device 16b has operation members 22i and 22j in addition to the operation members 22e, 22f, 22g and 22h shown in FIG. 4(b). While gripping the grip portion 21, the user operates the operation member 22i with the index finger of the right hand and operates the operation member 22j with the middle finger.
  • the input device 16a and the input device 16b will be referred to as the "input device 16" unless otherwise distinguished.
  • the operation member 22 provided in the input device 16 may be equipped with a touch sensing function that recognizes a finger just by touching it without pressing it.
  • the operating members 22f, 22g, 22j may comprise capacitive touch sensors.
  • the touch sensor may be mounted on another operation member 22, but when the input device 16 is placed on a mounting surface such as a table, the touch sensor is mounted on the operation member 22 that does not come into contact with the mounting surface. preferably.
  • the marker 30 is a light emitting portion that emits light to the outside of the case body 20.
  • the surface of the case body 20 includes a resin portion that diffuses and emits light from a light source such as an LED (Light Emitting Diode) element to the outside. .
  • the marker 30 is photographed by the imaging device 14 and used for tracking processing of the input device 16 .
  • the information processing device 10 uses the image captured by the imaging device 14 for tracking processing of the input device 16 and SLAM (Simultaneous Localization and Mapping) processing of the HMD 100 .
  • SLAM Simultaneous Localization and Mapping
  • the images captured at 120 frames/second by the imaging device 14 grayscale images captured at 60 frames/second are used for tracking processing of the input device 16, and other images captured at 60 frames/second are used. of full-color images may be used for concurrent HMD 100 self-localization and environment mapping.
  • FIG. 6 shows an example of part of an image of the input device 16 captured.
  • This image is an image of the input device 16b held by the right hand, and includes images of a plurality of markers 30 emitting light.
  • the communication control unit 128 transmits image data captured by the imaging device 14 to the information processing device 10 in real time.
  • FIG. 7 shows functional blocks of the input device 16.
  • the control unit 50 receives operation information input to the operation member 22 .
  • the control unit 50 also receives sensor data detected by the IMU (inertial measurement unit) 32 and sensor data detected by the touch sensor 24 .
  • the touch sensor 24 is attached to at least a portion of the plurality of operation members 22 and detects a state in which the user's finger is in contact with the operation member 22 .
  • the IMU 32 acquires sensor data regarding the movement of the input device 16 and includes an acceleration sensor 34 that detects at least three-axis acceleration data and an angular velocity sensor 36 that detects three-axis angular velocity data. Acceleration sensor 34 and angular velocity sensor 36 detect the value (sensor data) of each axial component at a predetermined cycle (800 Hz, for example).
  • the control unit 50 supplies the received operation information and sensor data to the communication control unit 54, and the communication control unit 54 transmits the operation information and sensor data to the information processing apparatus 10 by wired or wireless communication via a network adapter or an antenna. Send to
  • the input device 16 includes multiple light sources 58 for lighting multiple markers 30 .
  • Light source 58 may be an LED element that emits light in a predetermined color.
  • the control unit 50 causes the light source 58 to emit light based on the light emission instruction, and lights the marker 30 .
  • one light source 58 is provided for one marker 30 in the example shown in FIG. 7 , one light source 58 may light a plurality of markers 30 .
  • the vibrator 52 presents the user with tactile stimulation for game production.
  • the information processing apparatus 10 transmits vibration instructions to the input device 16 as the game progresses.
  • the control unit 50 causes the vibrator 52 to vibrate based on the vibration instruction.
  • the vibrator 52 provides the user with a tactile sensation corresponding to the progress of the game, thereby enhancing the user's sense of immersion in the game.
  • Vibrator 52 may be, for example, a voice coil motor.
  • FIG. 8 shows functional blocks of the information processing device 10 .
  • the information processing apparatus 10 includes a processing unit 200 and a communication unit 202.
  • the processing unit 200 includes an acquisition unit 210, a game execution unit 220, an image signal processing unit 222, a marker information holding unit 224, a state holding unit 226, and an estimation processing unit. 230 , a contact determination unit 232 , a stationary determination unit 234 , an image signal processing unit 268 and a SLAM processing unit 270 .
  • the communication unit 202 receives operation information of the operation member 22 and sensor data transmitted from the input device 16 and supplies the information to the acquisition unit 210 .
  • the communication unit 202 also receives captured image data and sensor data transmitted from the HMD 100 and supplies the data to the acquisition unit 210 .
  • the acquisition unit 210 includes a captured image acquisition unit 212 , a sensor data acquisition unit 214 and an operation information acquisition unit 216 .
  • the information processing device 10 includes a computer, and various functions shown in FIG. 8 are realized by the computer executing programs.
  • a computer includes, as hardware, a memory for loading a program, one or more processors for executing the loaded program, an auxiliary storage device, and other LSIs.
  • a processor is composed of a plurality of electronic circuits including semiconductor integrated circuits and LSIs, and the plurality of electronic circuits may be mounted on one chip or may be mounted on a plurality of chips.
  • the functional blocks shown in FIG. 8 are realized by cooperation of hardware and software, and therefore those skilled in the art will understand that these functional blocks can be realized in various forms by hardware alone, software alone, or a combination thereof. It is understood.
  • the captured image acquisition unit 212 acquires a full-color image for SLAM processing of the HMD 100 and supplies it to the image signal processing unit 268 .
  • the image signal processing unit 268 performs image signal processing such as noise reduction and optical correction (shading correction) on the image data, and supplies the image data subjected to the image signal processing to the SLAM processing unit 270 .
  • the sensor data acquisition unit 214 acquires sensor data transmitted from the HMD 100 and supplies it to the SLAM processing unit 270.
  • the SLAM processing unit 270 simultaneously executes self-position estimation and environment map creation of the HMD 100 based on the image data supplied from the captured image acquisition unit 212 and the sensor data supplied from the sensor data acquisition unit 214 .
  • the captured image acquisition unit 212 acquires a grayscale image for tracking processing of the input device 16 and supplies it to the image signal processing unit 222 .
  • the image signal processing unit 222 performs image signal processing such as noise reduction and optical correction (shading correction) on the image data, and supplies the image data subjected to the image signal processing to the first estimation processing unit 240 .
  • the first estimation processing unit 240 includes a marker image coordinate specifying unit 242, a position/orientation derivation unit 244, and a noise derivation unit 246, and estimates the position and orientation of the input device 16 based on the captured image of the input device 16. Realize the estimation function.
  • the first estimation processing unit 240 extracts marker images obtained by photographing the plurality of markers 30 of the input device 16 from the photographed image, and estimates the position and orientation of the input device 16 from the arrangement of the plurality of extracted marker images.
  • the first estimation processing section 240 outputs the estimated position and orientation of the input device 16 to the third estimation processing section 260 together with the noise (error) variance.
  • the sensor data acquisition unit 214 acquires sensor data transmitted from the input device 16 and supplies the sensor data to the second estimation processing unit 250 .
  • the second estimation processing unit 250 implements a second estimation function of estimating the velocity, position and orientation of the input device 16 based on sensor data indicating the acceleration and angular velocity of the input device 16 .
  • the second estimation function is a function that performs a state prediction step in the Kalman filter, and the second estimation processing unit 250 applies the supplied sensor data to the state vector (velocity, position, attitude) at the previous time.
  • the state vector at the current time is estimated by adding the amount of change in the state vector obtained by the integration operation.
  • the second estimation processing section 250 outputs the estimated state vector together with the noise variance to the third estimation processing section 260 . Note that the amount of change obtained by the integral calculation accumulates noise over time. ) tends to move away from
  • the third estimation processing unit 260 From the position and orientation of the input device 16 estimated by the first estimation processing unit 240 and the state vector (velocity, position, orientation) of the input device 16 estimated by the second estimation processing unit 250, the third estimation processing unit 260: The velocity, position and attitude of the input device 16 are derived with high accuracy.
  • the third estimation processing unit 260 may perform a filtering step (correction step) of UKF (unscented Kalman filter).
  • the third estimation processing unit 260 acquires the state vector estimated by the second estimation processing unit 250 as the “preliminary estimation value”, and acquires the position and orientation estimated by the first estimation processing unit 240 as the “observed values”.
  • the Kalman gain is calculated, and the "posterior estimate” is obtained by correcting the “preliminary estimate” using the Kalman gain.
  • the “posterior estimate” expresses the speed, position, and orientation of the input device 16 with high precision, is provided to the game execution unit 220, is recorded in the state holding unit 226, and is used as the next value in the second estimation processing unit 250. is used to estimate the state vector at the time of
  • a method of improving accuracy by integrating analysis results using multiple sensors such as the imaging device 14 and IMU 32 is known as sensor fusion.
  • sensor fusion it is necessary to express the time when data is acquired by each sensor on a common time axis.
  • the imaging cycle of the imaging device 14 and the sampling cycle of the IMU 32 are different and asynchronous.
  • the estimation processing unit 260 can estimate the position and orientation of the input device 16 with high accuracy.
  • the operation information acquisition section 216 acquires operation information transmitted from the input device 16 and supplies it to the game execution section 220 .
  • the game execution unit 220 progresses the game based on the operation information and the position/orientation information of the input device 16 estimated by the estimation processing unit 230 .
  • FIG. 9 is a flowchart showing position and orientation estimation processing by the first estimation processing unit 240.
  • the photographed image acquisition unit 212 acquires image data obtained by photographing the input device 16 ( S ⁇ b>10 ), and supplies the image data to the image signal processing unit 222 .
  • the image signal processing unit 222 applies image signal processing such as noise reduction and optical correction to the image data (S12), and supplies the image data subjected to the image signal processing to the marker image coordinate specifying unit 242.
  • the marker image coordinate specifying unit 242 specifies representative coordinates of a plurality of marker images included in the captured image (S14). When the brightness of each pixel of the grayscale image is represented by 8 bits and has a brightness value of 0 to 255, the marker image is captured as a high brightness image as shown in FIG.
  • the marker image coordinate specifying unit 242 specifies an area in which pixels having a luminance value equal to or greater than a predetermined value (for example, 128 luminance values) are continuous from the captured image, calculates the barycentric coordinates of the continuous pixel area, and determines the marker image. Representative coordinates may be specified.
  • the captured image includes not only the marker image, but also the image of lighting equipment such as electric lights. Therefore, the marker image coordinate specifying unit 242 checks several predetermined criteria to check whether a continuous pixel region having a brightness value equal to or greater than a predetermined value corresponds to the marker image. For example, if the continuous pixel area is too large or has a long shape, it is certain that the continuous pixel area does not correspond to the marker image. is not a marker image.
  • the marker image coordinate specifying unit 242 calculates the coordinates of the center of gravity of the continuous pixel region that satisfies a predetermined criterion, specifies the representative coordinates of the marker image (marker image coordinates), and stores the specified representative coordinates in a memory (not shown).
  • the marker information holding unit 224 holds the three-dimensional coordinates of each marker in the three-dimensional model of the input device 16 in the reference position and the reference posture.
  • a method of solving a PNP (Perspective n-Point) problem is known as a method of estimating the position and orientation of an imaging device that has taken an image of an object whose three-dimensional shape and size are known.
  • the position/orientation deriving unit 244 reads out N (N is an integer of 3 or more) marker image coordinates from a memory (not shown), and combines the read N marker image coordinates with the three-dimensional coordinates of the input device 16 .
  • the position and orientation of the input device 16 are estimated from the three-dimensional coordinates of the N markers in the model.
  • the position and orientation derivation unit 244 estimates the position and orientation of the imaging device 14 using the following (Equation 1), and derives the three-dimensional space position and orientation of the input device 16 based on the estimation result.
  • (u, v) are the marker image coordinates in the captured image
  • (X, Y, Z) are the three-dimensional space of the marker 30 when the three-dimensional model of the input device 16 is in the reference position and orientation.
  • is the position coordinates of The three-dimensional model has exactly the same shape and size as the input device 16, and the markers are arranged at the same positions. It holds the three-dimensional coordinates of each marker.
  • the position/orientation deriving unit 244 reads out the three-dimensional coordinates of each marker from the marker information holding unit 224 and obtains (X, Y, Z).
  • (f x , f y ) is the focal length of the imaging device 14 and (c x , c y ) is the image principal point, both of which are internal parameters of the imaging device 14 .
  • a matrix whose elements are r 11 to r 33 and t 1 to t 3 is a rotation/translation matrix.
  • (u, v), (f x , f y ), (c x , c y ), and (X, Y, Z) are known in (Equation 1), and the position/orientation derivation unit 244 calculates N markers Solving the equations for 30 gives their common rotation-translation matrix.
  • processing for estimating the position and orientation of the input device 16 is performed by solving a P3P problem.
  • the position/orientation derivation unit 244 extracts arbitrary three marker image coordinates from the plurality of marker image coordinates specified by the marker image coordinate specifying unit 242 .
  • the position/orientation derivation unit 244 reads out the three-dimensional coordinates of the markers in the three-dimensional model from the marker information storage unit 224 and solves the P3P problem using (Equation 1). After identifying the rotation/translation matrix common to the three extracted marker image coordinates, the position/orientation deriving unit 244 uses the marker image coordinates of the input device 16 other than the extracted three marker image coordinates to calculate the reprojection error. Calculate
  • the position/orientation derivation unit 244 extracts a predetermined number of combinations of three marker image coordinates.
  • the position/orientation deriving unit 244 identifies a rotation/translation matrix for each combination of the extracted three marker image coordinates, and calculates each reprojection error. Then, the position/orientation derivation unit 244 identifies the rotation/translation matrix with the smallest reprojection error from among the predetermined number of reprojection errors, and derives the position and orientation of the input device 16 (S16).
  • the position/orientation derivation unit 244 derives the world coordinate position and orientation of the input device 16 determined by combining the position and orientation of the input device 16 estimated in the HMD coordinate system and the world coordinate position and orientation of the HMD 100. do.
  • the noise derivation unit 246 derives the variance of the noise (error) of each of the estimated positions and orientations (S18).
  • the variance value of the noise corresponds to the reliability of the estimated position and orientation: the higher the confidence, the smaller the variance, and the lower the confidence, the larger the variance.
  • the noise derivation unit 246 may derive the variance of noise based on the distance between the imaging device 14 and the input device 16 and the position of the marker image within the angle of view. For example, when the imaging device 14 and the input device 16 are far apart or extremely close, or when the marker image is positioned at the edge of the captured image, it is difficult to derive the correct barycentric coordinates of the marker image. , the noise variance tends to be large.
  • the position/orientation estimation processing by the first estimation processing unit 240 is performed at the tracking image capturing cycle (60 frames/second) of the input device 16 (N in S20).
  • the position/orientation estimation processing by the first estimation processing unit 240 ends (Y in S20).
  • FIG. 10 shows the internal configuration of the estimation processing unit 230.
  • first estimation processing section 240 outputs the estimated position and orientation to third estimation processing section 260 as “observation value n k ” and the variance of position noise and orientation noise as “observation noise R k ”.
  • Observed value nk Observation vector at time k
  • Observation noise Rk Error covariance matrix of observed value at time k
  • the second estimation processing unit 250 reads out the “state vector m k ⁇ 1
  • the state variable m in the embodiment includes velocity, position, and attitude of the input device 16, but may also include acceleration bias and angular velocity bias.
  • k ⁇ 1 State vector at time k ⁇ 1 estimated with information up to time k ⁇ 1
  • k ⁇ 1 Estimated with information up to time k ⁇ 1 estimated error covariance matrix of the state at time k-1
  • the second estimation processing unit 250 acquires the acceleration a k and the angular velocity ⁇ k of the input device 16 from the sensor data acquisition unit 214, and sends the acceleration a k and the angular velocity ⁇ k as the “process input l k ” to the prediction unit. input.
  • Acceleration a k Acceleration at time k
  • Angular velocity ⁇ k Angular velocity at time k
  • Process input l k Process input vector at time k
  • the second estimation processing unit 250 calculates the variance of the acceleration noise and the variance of the angular velocity noise from the acceleration ak , the angular velocity ⁇ k , and fixed noise parameters (including axis deviation, scale deviation, value deviation, and bias deviation). , “process noise Q k ” to the predictor. ⁇ Process noise Q k : Error covariance matrix of process input at time k
  • the prediction unit integrates the acceleration a k and the angular velocity ⁇ k , respectively, and calculates the amount of change from the "state vector m k ⁇ 1
  • the prediction unit integrates the acceleration a k to calculate the amount of change in speed, and adds the calculated amount of change in speed to the speed at time k ⁇ 1 included in the “state vector m k ⁇ 1
  • the prediction unit integrates the estimated velocity at time k to calculate the amount of position change, and adds the calculated amount of position change to the position at time k ⁇ 1 included in “state vector m k ⁇ 1
  • the prediction unit also integrates the angular velocity ⁇ k to calculate the posture change amount, and adds the calculated posture change amount to the posture at time k ⁇ 1 included in the “state vector m k ⁇ 1
  • the prediction unit calculates the "state vector m k
  • the prediction unit outputs the “state vector m k
  • k ⁇ 1 State vector at time k estimated from information up to time k ⁇ 1
  • k ⁇ 1 State at time k estimated from information up to time k ⁇ 1 estimated error covariance matrix
  • the third estimation processing unit 260 acquires the “observation value n k ” and the “observation noise R k ” from the first estimation processing unit 240, and acquires the “state vector m k
  • the third estimation processing unit 260 corrects the “state vector m k
  • k State vector at time k estimated from information up to time k
  • k Estimation error covariance matrix of state at time k estimated from information up to time k
  • the state vector estimated by the second estimation processing unit 250 contains integral drift, and the amount of drift is accumulated each time the integral calculation is repeated.
  • the third estimation processing unit 260 uses the highly reliable “observed value n k ” at the cycle (60 Hz) of the tracking processing in the first estimation processing unit 240, and the second estimation processing unit 250 By correcting the state vector (velocity, position, attitude) output by , a highly accurate “state vector m k
  • k ” includes velocity, position, and orientation in world coordinates, and may be provided to the game execution section 220 and used for game operations.
  • k ” are temporarily held in the state holding unit 226 and read out during the estimation processing at time k+1 in the second estimation processing unit 250 .
  • the estimation processing by the first estimation processing unit 240 is performed at a cycle of 60 Hz, while the estimation processing by the second estimation processing unit 250 is performed at a cycle of 800 Hz. Therefore, the second estimation processing unit 250 successively updates the state vector after the first estimation processing unit 240 outputs an observed value until it outputs the next observed value, and the state vector is not corrected during this period.
  • the estimation processing unit 230 of the embodiment performs a correction step based on the state at time k ⁇ 1 immediately before observation time k, that is, the observed value is used to correct the past state.
  • the world coordinate position of the input device 16 is determined by combining the position of the input device 16 estimated in the HMD coordinate system of the HMD 100 and the world coordinate position of the HMD 100 . Since the world coordinate system generated by SLAM contains errors, and the estimation processing in the estimation processing unit 230 also contains errors, the estimated world coordinate position of the input device 16 always contains errors.
  • the estimation processing unit 230 performs a position fixing process of fixing and not changing the world coordinate position of the input device 16 when it is certain that the input device 16 is stationary.
  • the processing unit 200 includes a contact determination unit 232 and a stationary determination unit 234 to reliably determine that the input device 16 is stationary.
  • the stationary determination unit 234 determines whether the input device 16 is stationary based on sensor data indicating acceleration and/or angular velocity of the input device 16 .
  • the stationary determination unit 234 may determine whether or not the input device 16 is stationary based on sensor data indicating angular velocity, for example.
  • the stationary determination unit 234 determines that the input device 16 is stationary when the angular velocity is continuously equal to or less than the predetermined threshold for the predetermined period of time. If not, it is determined that the input device 16 is moving.
  • the time for determining whether the vehicle is stationary may be 1.5 seconds, and the predetermined threshold may be set to 3 deg/second, for example.
  • the stationary determination unit 234 may determine that the input device 16 is stationary with an additional condition that the variance of the latest 10 samples is 1 deg/sec or less. Determining that the input device 16 is still constitutes a first condition for the estimation processing unit 230 to perform the position fixation processing.
  • the contact determination unit 232 determines whether or not the user is touching the input device 16 .
  • the contact determination section 232 may determine whether or not the user is touching the input device 16 based on sensor data from the touch sensor 24 provided in the input device 16 . Determining that the user is not touching the input device 16 constitutes a second condition for the estimation processing unit 230 to perform the position fixation processing.
  • the position fixing unit 262 detects that the input device 16 is surely stationary when both the first condition and the second condition are satisfied at the same time. According to experiments conducted by the present disclosure, it has been found that a stationary determination time of about 10 seconds is required to reliably detect that the input device 16 is stationary only under the first condition. The present disclosure person has obtained knowledge that adding the second condition can shorten the stationary determination time (for example, 1.5 seconds) under the first condition. By establishing simultaneously, the stationary state is reliably detected in a short period of time.
  • the position fixing section 262 performs position fixing processing for fixing the world coordinate position P of the input device 16 estimated by the third estimation processing section 260 .
  • the position fixing unit 262 prevents the third estimation processing unit 260 from deriving a new estimated position during the position fixing process. Since the estimated position is not updated, the position fixing unit 262 may stop the execution of the state prediction step by the second estimation processing unit 250 and the correction step by the third estimation processing unit 260 .
  • the position fixing unit 262 performing position fixing processing, it is possible to avoid situations in which the position of the completely stationary input device 16 fluctuates in the world coordinate system.
  • the position fixing unit 262 when the contact determination unit 232 determines that the user is touching the input device 16 or the rest determination unit 234 determines that the input device 16 is moving, the position fixing unit 262 , the execution of the position fixation process is terminated, and the fixation of the position of the input device 16 is released.
  • the position fixing unit 262 restarts the operations of the second estimation processing unit 250 and the third estimation processing unit 260 to perform normal position/orientation estimation processing.
  • the third estimation processing unit 260 uses the position and orientation of the input device 16 estimated by the first estimation processing unit 240 and the state vector of the input device 16 estimated by the second estimation processing unit 250. , a process of estimating the velocity, position and orientation of the input device 16 with high accuracy is performed.
  • the fixed position of the input device 16 is instantly updated to the position estimated by the position/orientation estimation process. Therefore, if there is a large deviation (distance in the world coordinate system) between the world coordinate position estimated by the third estimation processing unit 260 and the fixed world coordinate position, the user will display the input device 16 on the game screen. It feels like the corresponding game object teleports to another position. Therefore, in preparation for resuming normal position/orientation estimation processing, it is preferable to keep the deviation (distance) between the estimated world coordinate position and the fixed position as small as possible.
  • the position estimated by the first estimation processing unit 240 and the fixed position It operates to keep the deviation (distance) from the position small.
  • the position fixing unit 262 monitors the relationship between the position estimated by the first estimation processing unit 240 and the fixed position.
  • the position of the device 16 is released and normal position and orientation estimation processing resumes.
  • the position fixing unit 262 monitors the distance between the position estimated by the first estimation processing unit 240 and the fixed position, and when the two positions are separated by a predetermined distance (Dth) or more, the position fixing process is executed.
  • the process is temporarily interrupted, the position of the input device 16 is released, and normal position/orientation estimation processing is resumed.
  • the predetermined distance Dth is determined according to the distance between the input device 16 and the imaging device 14, and the longer the distance between the input device 16 and the imaging device 14, the longer the predetermined distance Dth may be set. For example, when the distance between the input device 16 and the imaging device 14 is less than 1 m, the predetermined distance Dth is set to 5 cm, and when the distance between the input device 16 and the imaging device 14 is 1 m or more, the predetermined distance Dth is set to 20 cm. you can As the distance between the input device 16 and the imaging device 14 increases, the estimation accuracy of the first estimation processing unit 240 decreases. , to avoid the situation of frequently suspending the execution of the position fixing process.
  • the position fixing unit 262 After the normal position/orientation estimation process is resumed, when both the first condition and the second condition are satisfied at the same time, the position fixing unit 262 performs the position fixing process, and the third estimation processing unit 260 newly estimates the The world coordinate position P of the input device 16 is fixed. In this way, by adjusting the fixed position each time so that the difference (gap) between the estimated position and the fixed position does not become large, the game object corresponding to the input device 16 is moved when the normal position/orientation estimation processing is resumed. You can avoid situations where you move instantaneously.
  • FIG. 11 shows a flowchart of position fixing processing.
  • the position/orientation estimation process in which the first estimation processing unit 240, the second estimation processing unit 250, and the third estimation processing unit 260 cooperate is started (S30).
  • the position/orientation estimation process is performed at a cycle of 800 Hz (N of S32).
  • the stationary determination unit 234 determines whether the input device 16 is stationary based on sensor data indicating acceleration and/or angular velocity of the input device 16 (S34). If the angular velocity is not equal to or lower than the predetermined threshold continuously for the predetermined time, the stationary determination unit 234 determines that the input device 16 is moving (N of S34), and continues the position/orientation estimation process (S30). The stationary determination unit 234 determines that the input device 16 is stationary when the angular velocity is equal to or lower than a predetermined threshold continuously for a predetermined period of time (Y in S34). In order to determine that the input device 16 is stationary, an additional condition may be set that the variance of the latest 10 samples is 1 deg/sec or less.
  • the contact determination unit 232 determines whether or not the user is touching the input device 16 (S36). If the sensor data of the touch sensor 24 is a value indicating contact, the contact determination unit 232 determines that the user is touching (Y in S36), and continues the position/orientation estimation process (S30). The contact determination unit 232 determines that the user is not touching when the sensor data of the touch sensor 24 is a value indicating non-contact (N of S36).
  • the position fixing unit 262 fixes the world coordinate position P of the input device 16 estimated by the third estimation processing unit 260 (S38). At this time, the position fixing unit 262 stops the execution of the state prediction step by the second estimation processing unit 250 and the correction step by the third estimation processing unit 260, and maintains the fixed position coordinate P.
  • the position fixing unit 262 monitors the distance between the position estimated by the first estimation processing unit 240 and the fixed position P (S42).
  • the predetermined distance Dth is set according to the distance between the input device 16 and the imaging device 14, and the position fixing unit 262 specifies the distance between the input device 16 and the imaging device 14, and sets the distance Dth according to the distance.
  • the determined predetermined distance Dth and the distance between the estimated position and the fixed position P are compared. If the distance between the estimated position and the fixed position P is less than the predetermined distance Dth (N of S42), the process returns to S34 to determine whether the conditions for continuing the position fixing process are satisfied.
  • the position fixing unit 262 temporarily suspends the execution of the position fixing process in order to update the fixed position P. , restarts the position and orientation estimation processing (S30). After that, when the execution condition of the position fixing process (Y of S34, N of S36) is established, the position fixing unit 262 restarts the position fixing process and fixes the estimated position at that time (S38). When the game execution unit 220 ends the game, this flow ends (Y of S32, Y of S40).
  • the present disclosure has been described above based on the examples. It should be understood by those skilled in the art that the above embodiments are examples, and that various modifications can be made to combinations of each component and each treatment process, and such modifications are also within the scope of the present disclosure. .
  • the information processing apparatus 10 performs the estimation process in the embodiment, the functions of the information processing apparatus 10 may be provided in the HMD 100 and the HMD 100 may perform the estimation process. That is, the HMD 100 may be the information processing device 10 .
  • the imaging device 14 is attached to the HMD 100 in the embodiment, the imaging device 14 may be attached to a position other than the HMD 100 as long as it can capture the marker image.
  • the present disclosure can be used for techniques for estimating the location of a device.
  • Captured image acquisition unit 214 Sensor data acquisition unit 216 Operation information acquisition unit 220 Game execution unit 222 Image signal processing unit 224 Marker information storage unit 226 state holding unit 230 estimation processing unit 232 contact determination unit 234 stationary determination unit 240 first estimation processing unit 242 marker image coordinate specification Section 244 Position/orientation derivation section 246 Noise derivation section 250 Second estimation processing section 260 Third estimation processing section 262 Position fixation section 268 • Image signal processing unit 270 . . . SLAM processing unit.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • General Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Human Computer Interaction (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • User Interface Of Digital Computer (AREA)
  • Length Measuring Devices By Optical Means (AREA)
PCT/JP2022/030070 2022-02-18 2022-08-05 情報処理装置およびデバイス位置推定方法 Ceased WO2023157338A1 (ja)

Priority Applications (2)

Application Number Priority Date Filing Date Title
JP2024500932A JPWO2023157338A1 (https=) 2022-02-18 2022-08-05
US18/726,134 US12314486B1 (en) 2022-02-18 2022-08-05 Information processing apparatus and device position estimation method

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2022-023934 2022-02-18
JP2022023934 2022-02-18

Publications (1)

Publication Number Publication Date
WO2023157338A1 true WO2023157338A1 (ja) 2023-08-24

Family

ID=87577851

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/030070 Ceased WO2023157338A1 (ja) 2022-02-18 2022-08-05 情報処理装置およびデバイス位置推定方法

Country Status (3)

Country Link
US (1) US12314486B1 (https=)
JP (1) JPWO2023157338A1 (https=)
WO (1) WO2023157338A1 (https=)

Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012507101A (ja) * 2008-10-27 2012-03-22 株式会社ソニー・コンピュータエンタテインメント ボール付属コントローラの位置および移動の決定
JP2020513611A (ja) * 2016-11-15 2020-05-14 グーグル エルエルシー 仮想現実システムのための入力コントローラ安定化技術
JP2020181322A (ja) * 2019-04-24 2020-11-05 株式会社ソニー・インタラクティブエンタテインメント 情報処理装置および代表座標導出方法

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021240930A1 (ja) 2020-05-29 2021-12-02 株式会社ソニー・インタラクティブエンタテインメント 入力デバイス
US20250029328A1 (en) * 2023-07-23 2025-01-23 Apple Inc. Systems and methods for presenting content in a shared computer generated environment of a multi-user communication session

Patent Citations (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2012507101A (ja) * 2008-10-27 2012-03-22 株式会社ソニー・コンピュータエンタテインメント ボール付属コントローラの位置および移動の決定
JP2020513611A (ja) * 2016-11-15 2020-05-14 グーグル エルエルシー 仮想現実システムのための入力コントローラ安定化技術
JP2020181322A (ja) * 2019-04-24 2020-11-05 株式会社ソニー・インタラクティブエンタテインメント 情報処理装置および代表座標導出方法

Also Published As

Publication number Publication date
US20250155991A1 (en) 2025-05-15
JPWO2023157338A1 (https=) 2023-08-24
US12314486B1 (en) 2025-05-27

Similar Documents

Publication Publication Date Title
CN109146965B (zh) 信息处理装置、计算机可读介质和头戴式显示装置
CN110047104B (zh) 对象检测和跟踪方法、头戴式显示装置和存储介质
CN108139204B (zh) 信息处理装置、位置和/或姿态的估计方法及记录介质
JP7248490B2 (ja) 情報処理装置、デバイスの位置および姿勢の推定方法
WO2017077918A1 (ja) 情報処理装置、情報処理システム、および情報処理方法
JP7198149B2 (ja) 情報処理装置およびデバイス情報導出方法
WO2020189450A1 (ja) 複数のマーカを備えたデバイス
US12429959B2 (en) Information processing apparatus and device position acquisition method
JP7288792B2 (ja) 情報処理装置およびデバイス情報導出方法
WO2023238678A1 (ja) 情報処理装置、コントローラ表示方法およびコンピュータプログラム
US20240033618A1 (en) Information processing apparatus and image generating method
WO2023157338A1 (ja) 情報処理装置およびデバイス位置推定方法
JP7710386B2 (ja) 情報処理装置、デバイス速度推定方法およびデバイス位置推定方法
US12353648B2 (en) Information processing apparatus and device position estimation method
WO2023153094A1 (ja) 情報処理装置および代表座標導出方法
US20240257391A1 (en) Information processing apparatus and device information derivation method
US20260069982A1 (en) Information processing device and image generation method
US20250308061A1 (en) Information processing apparatus and representative coordinate derivation method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22927236

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 2024500932

Country of ref document: JP

WWE Wipo information: entry into national phase

Ref document number: 18726134

Country of ref document: US

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22927236

Country of ref document: EP

Kind code of ref document: A1

WWP Wipo information: published in national office

Ref document number: 18726134

Country of ref document: US

WWG Wipo information: grant in national office

Ref document number: 18726134

Country of ref document: US