Detecting Techniques for Mobile Interaction

1565 days ago, 567 views
PowerPoint PPT Presentation

Presentation Transcript

Slide 1

Ken Hinckley Jeff Pierce Mike Sinclair Eric Horvitz Attentional User Interfaces Project Microsoft Research Sensing Techniques for Mobile Interaction

Slide 2

Mobile Devices versus Mr. Cleo (An Informal Comparison) How would we be able to make Smart Computers? Mr. Cleo Aware of sounds, items Knows whether I stroll in the room Selfish & Inconsiderate My Mobile Device Unaware of environment Oblivious to my nearness Selfish & Inconsiderate

Slide 3

Sensitive Interfaces (a.k.a. "Not really Stupid Computing") Computers are quick boneheads. They are unmindful of the outer world. What is a "savvy PC" in any case??? Could we fabricate one? Do you truly need one? On the other hand one that knows, conscious, … Not So Stupid? We'd be in an ideal situation if PCs were as stupid as my feline Can some extremely moronic sensors, with direct programming, address this issue? Plan equipment/programming interfaces that are touchy to the client and the encompassing physical environment.

Slide 4

Sensing for User Interaction Sense more than simply unequivocal summons Simplify the interface utilizing foundation data that is as of now there Point+Shoot Cameras One Button Many sensors

Slide 5

Can "Foundation Sensing" Enhance Mobile UI? Genuine has high psychological/attentional request Even "clicking a catch" can be hard! Programming unmindful of the changing physical setting Device ought to adjust to current errand/circumstance Naturally happening "signals" of utilization are missed Pick up, put down, take a gander at, stroll around with… Explosion of modest, educational sensors On gadget or accessible through remote system

Slide 6

Mobile Sensor Prototype (Casio E105) Tilt Sensor Proximity Range Sensor Touch Sensor Demos actualized as Windows CE applications Sensor I/O by means of PIC smaller scale. VIDEO

Slide 7

Tilt Sensor Analog Devices 2-hub straight accelerometer Tilt with respect to gravity But, different increasing velocities likewise influence flag Limitations: Cannot detect revolution around vertical hub Cannot tell up from down

Slide 8

Activity Detection: Tilt Sensor Example Data Looking at show Walking to lift Walking to meeting PocketPC held at side

Slide 9

Voice Memo Detector Gesture permits universally useful PIM to have extraordinary reason setting of utilization Sensor Fusion: Must hold gadget Tilt "like a telephone" Hold near face Audio is pivotal to the association Relaxing quits recording

Slide 10

Normal approach Sensor approach Pick up gadget Find the ¼" dia. catch Position hand to squeeze catch Press & keep up strain Listen for beep Record your message Release when done Double-beep affirms Pick up gadget (to confront) Listen for the beep Record your message Relax gadget when done Double-beep affirms recording was made Voice Memo: Workflow Analysis "I need to consider finding the catch, pushing . it, holding it" "It was simply listen for the beep"

Slide 11

Sensed Memo Recording: Usability Issues Sensed signal not effortlessly discoverable "bewildering to set up to my face to talk" - 1 Ss False positives can happen: e.g. putting into sweater stash But, catch additionally has false positives… "talk into it like a mobile phone" is sufficient guideline to utilize "Significantly less demanding, I can center" "Would utilize it increasingly in the event that it worked that way" 6/7 Ss favored detected motion to catch (4.3)

Slide 12

Tracking Experiment Not speedier, yet less request on visual consideration. Client tracks "fly" on screen, utilizing mouse Records "Testing 123" (S, M) > C (p<.001) Sensed motion may require less visual consideration (p < 0.1)

Slide 13

Portrait/Landscape Display Mode Detection "Snow Globe" Input controls pivoted to match screen Easy (5.0/5.0). 6 Ss favor tilting to menu; 1 Ss "I think it would drive me insane" Sharing w/others Doesn't intrude

Slide 14

Portrait/Landscape Detection: Implementation Tilt versus detected introduction: 5 show modes 2 Portrait 2 Landscape Flat Dead groups, 0.5s: hold screen stable Put-down issue: FIFO line, search for stable situate.

Slide 15

Tilt for Scrolling Touch screen bezel to grasp Easy to initiate/keep up state Too simple to hit unintentionally (scene mode) Sets "resting" introduction Hides on-screen UI (menu, taskbar) amid looking over Several exchange capacities conceivable Rate ctrl: v = K * sgn (dA)  max (||dA|| - dA min , 0)  Single pivot: just move along 1 essential hub at once Dual hub: full panning in any bearing Mixed hub: panning, however with liking for essential tomahawks

Slide 16

Tilt: Scrolling & Then Some Contrast remuneration Equalize "obvious differentiation" Scrolling + Portrait/Landscape Don't change P/L modes while looking over! Try not to change when quit looking over, either! Holds up to see distinctive introduction Not exactly right; ought to switch after longer abide (2-3 s?) User Testing: 5 Ss, contrasted with implicit bearing cushion "Great approach to look over the screen" – Agree (4.8/5.0) "I'd rather utilize the course cushion" – Disagree (1.8/5.0) One-gave operation, regular, rearranges the development

Slide 17

Power Management Automatic power-on Holding gadget close by AND taking a gander at show (level in one hub, tilted fwd ~20 o in the other) for 0.5s Can't control up in tote/take Won't control up on the off chance that you simply touch it to push away Won't control up in the event that you simply get & hold at side (normally ) There is no auto-control OFF component, by outline Best case : kills & client doesn't see highlight Touch, tilt & vicinity sensors avoid control off or screen darkening while utilizing gadget

Slide 18

Other Fun Stuff Games Proximity For zooming – fun demo, yet how to choose? Proximal UI: Creepy "Hand of dissatisfaction" Shaking To switch applications (beat application ��  base) Turn topsy turvy & shake to delete :- ) Impl. not sufficiently strong to turn on constantly

Slide 19

Conclusions New UI's with better affectability to the client & the client's physical environs Great potential to improve & upgrade the UI ��  new practices and administrations that clients find convincing, valuable, connecting with, aware Design, Implementation, & Usability challenges… Design must handle false positive/false negative cases Not a panacea. Just appears to be useful for a few undertakings. Making things less complex versus Loss of unequivocal control Promising territory that requirements more work

Slide 21

Proximity Sensor IR LED, 60 o , 40kHz IR collector (utilized as a part of TV's) Gain �� distance Some light affectability, e.g. daylight

Slide 22

Touch Sensor Mainly so we know when client is holding gadget Useful to disambiguate "purposeful" signals from incidental ones holding and tilting, versus tilted while sitting in folder case Also explored different avenues regarding touch catches around screen bezel

Slide 23

Software Architecture "Setting Server" whiteboard Shared memory to peruse factors Or, request Windows messages to tell of changes Apps can post any integrated information back to server Some illustration setting factors (occasions): Holding, Duration TiltAngleLR/FB DisplayOrientation Walking Proximity (z) ProximityState Scrolling VoiceMemoGesture

Slide 24

Touch bezel for unequivocal control over device bars Proximity of hand to screen? helper UI shows up when hand draws near Screen Real Estate Optimization