Deprecated: The each() function is deprecated. This message will be suppressed on further calls in /home/zhenxiangba/zhenxiangba.com/public_html/phproxy-improved-master/index.php on line 456
NZ744300B2 - Systems and methods for augmented reality - Google Patents
[go: Go Back, main page]

NZ744300B2 - Systems and methods for augmented reality - Google Patents

Systems and methods for augmented reality Download PDF

Info

Publication number
NZ744300B2
NZ744300B2 NZ744300A NZ74430017A NZ744300B2 NZ 744300 B2 NZ744300 B2 NZ 744300B2 NZ 744300 A NZ744300 A NZ 744300A NZ 74430017 A NZ74430017 A NZ 74430017A NZ 744300 B2 NZ744300 B2 NZ 744300B2
Authority
NZ
New Zealand
Prior art keywords
fov
electromagnetic
camera
sensor
controller
Prior art date
Application number
NZ744300A
Other versions
NZ744300A (en
Inventor
David C Lundmark
Samuel A Miller
Michael J Woods
Original Assignee
Magic Leap Inc
Filing date
Publication date
Priority claimed from US15/062,104 external-priority patent/US20160259404A1/en
Application filed by Magic Leap Inc filed Critical Magic Leap Inc
Priority claimed from PCT/US2017/016722 external-priority patent/WO2017136833A1/en
Publication of NZ744300A publication Critical patent/NZ744300A/en
Publication of NZ744300B2 publication Critical patent/NZ744300B2/en

Links

Classifications

    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/211—Input arrangements for video game devices characterised by their sensors, purposes or types using inertial sensors, e.g. accelerometers or gyroscopes
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/21—Input arrangements for video game devices characterised by their sensors, purposes or types
    • A63F13/213—Input arrangements for video game devices characterised by their sensors, purposes or types comprising photodetecting means, e.g. cameras, photodiodes or infrared cells
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/20—Input arrangements for video game devices
    • A63F13/23—Input arrangements for video game devices for interfacing with the game device, e.g. specific interfaces between game controller and console
    • A63F13/235—Input arrangements for video game devices for interfacing with the game device, e.g. specific interfaces between game controller and console using a wireless connection, e.g. infrared or piconet
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/25—Output arrangements for video game devices
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/30—Interconnection arrangements between game servers and game devices; Interconnection arrangements between game devices; Interconnection arrangements between game servers
    • A63F13/35—Details of game servers
    • A—HUMAN NECESSITIES
    • A63—SPORTS; GAMES; AMUSEMENTS
    • A63F—CARD, BOARD, OR ROULETTE GAMES; INDOOR GAMES USING SMALL MOVING PLAYING BODIES; VIDEO GAMES; GAMES NOT OTHERWISE PROVIDED FOR
    • A63F13/00—Video games, i.e. games using an electronically generated display having two or more dimensions
    • A63F13/50—Controlling the output signals based on the game progress
    • A63F13/52—Controlling the output signals based on the game progress involving aspects of the displayed game scene
    • A63F13/525—Changing parameters of virtual cameras
    • A63F13/5255—Changing parameters of virtual cameras according to dedicated instructions from a player, e.g. using a secondary joystick to rotate the camera around a player's character
    • G—PHYSICS
    • G02—OPTICS
    • G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/01—Head-up displays
    • G02B27/017—Head mounted
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/03—Arrangements for converting the position or the displacement of a member into a coded form
    • G06F3/033—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor
    • G06F3/0346—Pointing devices displaced or positioned by the user, e.g. mice, trackballs, pens or joysticks; Accessories therefor with detection of the device orientation or free movement in a three-dimensional [3D] space, e.g. 3D mice, 6-DOF [six degrees of freedom] pointers using gyroscopes, accelerometers or tilt-sensors
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T19/00—Manipulating three-dimensional [3D] models or images for computer graphics
    • G06T19/006—Mixed reality

Abstract

Disclosed is an augmented reality (AR) display system, comprising a hand-held controller. The hand-held controller comprises an electromagnetic field emitter to emit a known magnetic field in a known coordinate system, and an inertial measurement (“IMU)” component that assists in determining positioning and/or orientation of the electromagnetic field emitter relative to other components. The augmented reality display system also comprises a head-worn display device comprising an electromagnetic sensor to measure a parameter related to a magnetic flux at the electromagnetic sensor resulting from the known magnetic field, wherein the electromagnetic field emitter and the electromagnetic sensor are mobile and a depth sensor to measure a distance between the head-worn display device and the hand-held controller in the known coordinate system. The augmented reality display also comprises a controller to determine pose information of the electromagnetic sensor relative to the electromagnetic field emitter in the known coordinate system based at least in part on the parameter related to the magnetic flux measured by the electromagnetic sensor and the distance measured by the depth sensor and a display system to display virtual content to a user based at least in part on the pose information of the electromagnetic sensor relative to the electromagnetic field emitter and a world capture camera and a picture camera. In the augmented reality display the depth sensor comprises a depth camera having a first field of view (FOV), the world capture camera has a second FOV at least partially overlapping with the first FOV, the picture camera has a third FOV at least partially overlapping with the first FOV and the second FOV, and the depth camera, the world capture camera, and the picture camera are configured to capture respective first, second, and third images. Further, the controller is programmed to segment the second and third images, and the controller is programmed to fuse the second and third images after segmenting the second and third images to generate a fused image.

Claims (4)

1. An augmented reality (AR) display system, comprising: a hand-held controller comprising: an electromagnetic field emitter to emit a known magnetic field in a known coordinate system, and an inertial measurement (“IMU)” component that assists in determining positioning and/or orientation of the electromagnetic field emitter relative to other components; a head-worn display device comprising an electromagnetic sensor to measure a parameter related to a magnetic flux at the electromagnetic sensor resulting from the known magnetic field, wherein the electromagnetic field emitter and the electromagnetic sensor are mobile; a depth sensor to measure a distance between the head-worn display device and the hand- held controller in the known coordinate system; a controller to determine pose information of the electromagnetic sensor relative to the electromagnetic field emitter in the known coordinate system based at least in part on the parameter related to the magnetic flux measured by the electromagnetic sensor and the distance measured by the depth sensor; and a display system to display virtual content to a user based at least in part on the pose information of the electromagnetic sensor relative to the electromagnetic field emitter; and a world capture camera and a picture camera, wherein the depth sensor comprises a depth camera having a first field of view (FOV), wherein the world capture camera has a second FOV at least partially overlapping with the first FOV, wherein the picture camera has a third FOV at least partially overlapping with the first FOV and the second FOV, wherein the depth camera, the world capture camera, and the picture camera are configured to capture respective first, second, and third images, wherein the controller is programmed to segment the second and third images, and wherein the controller is programmed to fuse the second and third images after segmenting the second and third images to generate a fused image.
2. The AR display system of claim 1, further comprising an additional localization resource to provide additional information, wherein the pose information of the electromagnetic sensor relative to the electromagnetic field emitter in the known coordinate system is determined based at least in part on the parameter related to the magnetic flux measured by the electromagnetic sensor, the distance measured by the depth sensor, and the additional information provided by the additional localization resource.
3. The AR display system of either of claim 1 or 2, wherein the electromagnetic sensor is attached to a head set of the AR display system.
4. An augmented reality display system, comprising: a hand-held component comprising an electromagnetic field emitter, and an inertial measurement (“IMU)” component that assists in determining positioning and/or orientation of the electromagnetic field emitter relative to other components, the electromagnetic field emitter emitting a magnetic field; a head-mounted component having a display system that displays virtual content to a user, the head mounted component coupled to an electromagnetic sensor measuring a parameter related to a magnetic flux at the electromagnetic sensor resulting from the magnetic field, wherein a head pose of the head-mounted component in a known coordinate system is known, wherein the electromagnetic field emitter and the electromagnetic sensor are mobile; a depth sensor measuring a distance between the head-worn component and the hand- held component in the known coordinate system; a controller communicatively coupled to the hand-held component, the head-mounted component, and the depth sensor, the controller receiving the parameter related to the magnetic flux at the electromagnetic sensor from the head mounted component and receiving the distance from the depth sensor; and a world capture camera and a picture camera, wherein the controller determines a hand pose of the hand-held component based at least in part on the parameter related to the magnetic flux measured by the electromagnetic sensor and the distance measured by the depth sensor, wherein the system modifies the virtual content displayed to the user based at least in part on the hand pose, wherein the depth sensor comprises a depth camera having a first field of view (FOV), wherein the world capture camera has a second FOV at least partially overlapping with the first FOV, wherein the picture camera has a third FOV at least partially overlapping with the first FOV and the second FOV, wherein the depth camera, the world capture camera, and the picture camera are configured to capture respective first, second, and third images wherein the controller is programmed to segment the second and third images, and wherein the controller is programmed to fuse the second and third images after segmenting the second and third images to generate a fused image.
NZ744300A 2017-02-06 Systems and methods for augmented reality NZ744300B2 (en)

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US201662292185P 2016-02-05 2016-02-05
US201662298993P 2016-02-23 2016-02-23
US15/062,104 US20160259404A1 (en) 2015-03-05 2016-03-05 Systems and methods for augmented reality
PCT/US2017/016722 WO2017136833A1 (en) 2016-02-05 2017-02-06 Systems and methods for augmented reality

Publications (2)

Publication Number Publication Date
NZ744300A NZ744300A (en) 2024-08-30
NZ744300B2 true NZ744300B2 (en) 2024-12-03

Family

ID=

Similar Documents

Publication Publication Date Title
TWI496108B (en) AR image processing apparatus and method
US10769802B2 (en) Indoor distance measurement method
KR102079097B1 (en) Device and method for implementing augmented reality using transparent display
CN112446906B (en) Video overlay display registered in virtual space
IL292427B2 (en) Imaging modification, display and visualization using augmented and virtual reality eyewear
WO2019152617A1 (en) Calibration system and method to align a 3d virtual scene and 3d real world for a stereoscopic head-mounted display
JP6295296B2 (en) Complex system and target marker
US20150161762A1 (en) Information processing apparatus, information processing method, and program
JP2002092647A (en) Information presentation system and model error detection system
US20200081249A1 (en) Internal edge verification
US11024040B2 (en) Dynamic object tracking
WO2014175323A1 (en) Part attachment work support system and part attachment method
US10539790B2 (en) Coordinate matching apparatus for head-up display
JP7414395B2 (en) Information projection system, control device, and information projection control method
CN108132490A (en) Detection system and detection method based on alignment system and AR/MR
IL274205B2 (en) A system and method for multi-level viewing
WO2019193859A1 (en) Camera calibration method, camera calibration device, camera calibration system and camera calibration program
JPH06189906A (en) Gaze direction measuring device
CN104121892B (en) Method, device and system for acquiring light gun shooting target position
JP2021536605A5 (en)
NZ744300B2 (en) Systems and methods for augmented reality
NZ744300A (en) Systems and methods for augmented reality
CN108627157A (en) A kind of head based on three-dimensional marking plate shows localization method, device and three-dimensional marking plate
JP2010079459A (en) Indicator system
US10634920B2 (en) System and method for recording and representing measurement points on a body surface