Gyroscope! control for mouse.

[Screasey]

This seems to make a lot of sense. Obviously this would only be for users with iPhone 4, but if done right, would be easier than using the accelerometer.

[MobileMouseAdmin]

It's already in the newest version, 2.1.0


This seems to make a lot of sense. Obviously this would only be for users with iPhone 4, but if done right, would be easier than using the accelerometer.

[jason98]

Hi Guys,

First of all, thank you for the really great app.

I have a question regarding gyroscope control (yes, I am using iPhone 4)
It seems like the mouse pointer quickly looses its center location after a few movements.

As far as I understand, with a gyroscope whenever I point the device into the original direction I expect the mouse pointer to return to the original position where the gyroscope was engaged.
But that's not definitely a case with the Air Mouse.
After just a few moves, I have to recenter the gyroscope and it is really annoying.

Could you please clarify whether it's intentional behavior, a bug on your side, or a hardware limitation.
I do not think that it is a hardware problem though, because in the game called Gun Range, once you center the gyroscope, it never seem to loose the original direction.

Thanks!

[MobileMouseAdmin]

The in-air mouse movement works relative to where you tap the trigger button. The reason for this is so you can move longer ranges of distance. So if you start your movement then reach the limit of how much you can turn your hand, you can take your finger off the trigger button, then move you hand back to the center position then tap the trigger button and continue moving your hand. This is similar to when you reach the edge of a mouse pad with your physical mouse and you have to pick up the move and move it back to the center of the pad.


Hi Guys,

First of all, thank you for the really great app.

I have a question regarding gyroscope control (yes, I am using iPhone 4)
It seems like the mouse pointer quickly looses its center location after a few movements.

As far as I understand, with a gyroscope whenever I point the device into the original direction I expect the mouse pointer to return to the original position where the gyroscope was engaged.
But that's not definitely a case with the Air Mouse.
After just a few moves, I have to recenter the gyroscope and it is really annoying.

Could you please clarify whether it's intentional behavior, a bug on your side, or a hardware limitation.
I do not think that it is a hardware problem though, because in the game called Gun Range, once you center the gyroscope, it never seem to loose the original direction.

Thanks!

[jason98]

The in-air mouse movement works relative to where you tap the trigger button.

I guess I was not clear with my point. I understand that gyroscope is relative to where i tap and I can retap to keep moving.

The problem I am describing here is reproducible when you tap once and then continuously move the phone in different directions and then return it to the original direction of the tap.
At this point mouse pointer is NOT in the original location on the screen!

I can't believe you have not heard this problem before, maybe number of your iPhone 4 users is still quite low... But here are the step by step instructions to reproduce it:

1. Move pc mouse to the center of your pc screen
2. Orient iPhone towards the center of the pc screen.
3. Tap to engage the gyroscope
4. While continuosly holding the tap, move iPhone in different directions: left, right, up, or down
5. While still holding the tap repoint iPhone back to the center
6. Look where mouse pointer is. It is expected to return back to the center but it is NOT!

In fact this behavior is so annoying that I am willing to return the app if there is no plans to fix it.

[MobileMouseAdmin]

This is due to the acceleration feature. The mouse moves a greater distance the faster you move it. If you don't want this feature enabled, go into the server settings and turn down the acceleration slider in the Air Motion tab.


I guess I was not clear with my point. I understand that gyroscope is relative to where i tap and I can retap to keep moving.

The problem I am describing here is reproducible when you tap once and then continuously move the phone in different directions and then return it to the original direction of the tap.
At this point mouse pointer is NOT in the original location on the screen!

I can't believe you have not heard this problem before, maybe number of your iPhone 4 users is still quite low... But here are the step by step instructions to reproduce it:

1. Move pc mouse to the center of your pc screen
2. Orient iPhone towards the center of the pc screen.
3. Tap to engage the gyroscope
4. While continuosly holding the tap, move iPhone in different directions: left, right, up, or down
5. While still holding the tap repoint iPhone back to the center
6. Look where mouse pointer is. It is expected to return back to the center but it is NOT!

In fact this behavior is so annoying that I am willing to return the app if there is no plans to fix it.

[jason98]

This is due to the acceleration feature. The mouse moves a greater distance the faster you move it. If you don't want this feature enabled, go into the server settings and turn down the acceleration slider in the Air Motion tab.

Well, I turned it all the way down and the issue is still present.
I also just noticed that partly the problem is due to the fact when the mouse pointer reaches an edge of the screen it stops but you keep moving the gyroscope outside of the screen until let's say it points 90 degrees from the tap point (that is parallel to the screen.) Then you start moving in opposit direction back towards the screen and the pointer immediately responds although the iPhone is still pointing almost parallel to the screen.
But again the problem is still there even if you do not reach screen edges.

So here is how the gyroscope mouse should work:
Once the tap is made the mapping between polar coordinated of the gyroscope and 2d coordinates of the screen must be defined and be stable regardless of whether the mouse happens to reach an edge of the screen or how hard you rotate the gyroscope device.

To be clear here is the use case:
Say you have a big projector screen and you make a presentation using iPhone 4 as a gyroscope mouse device. You stand at a certain distance from the screen and you should be able to adjust mouse sensitivity so that when you point iPhone to the screen the mouse moves to the same spot...
Ideally there should be no need to hold the tap down while you control the mouse in this case and of course it is absolutely not acceptable to desynchronize mouse position and gyroscope direction. Even when you make a full 360 degree rotation around yourself the mouse pointer must leave the screen at one side of the screen and then reenter at another side of it when you complete your rotation.
Hope this explanation helps...

[Sixten]

Hi I agree with jason here. I just bought Airmouse and I find it too hard to use.
I have used it on my 3 GS but I will get iPhone 4 tomorrow so I haven't tried the gyroscope feature yet.
Still about gyroscope use:
I had hoped that Airmouse would be "locked" to the screen. Then I think it would be almost just like a wiimote.
I don't see the need for the acceleration feature -the cursor should just go wherever you point the iPhone
without having to hold any trigger/tab button like jason mentions.
But even better than a Wii mote it should possible to specify a range so that with a wide range the cursor goes to the edges of the screen only when the iPhone is nearly sideways (relative to original position). That would make a very precise pointer I believe. Ofcourse in everyday use you would set a less wide range.
Please make Airmouse like this! Then it will be fantastic!

[juddadredd]

Hi sixten,

Im going to try to help out on this one for you, if they tried locking the mouse in air mode to the screen they would have to have
1. An IR receiver next to your monitor
2. An IR transmitter on the iPhone

The wii reads the signal from the two devices and then with some really clear dampening code averages out roughly were the mouse pointer will go.

Jason makes a great argument but I guess he isn't an arm talker like I am, for the last 3 years or so so i do know a thing or two about the software, now when I'm giving a presentation I use my projector and my air mouse, I don't work for air mouse but I have used their software from the first day it was released and I didn't get vocal on the subject until I got my iPad version. Now to get my point over to the people in my meeting I wave my arms around and between slides my arms go by my sides, now thats the reason why you have to hold the button to move the air mouse... To stop excess user input if it was on all the time everytime I looked away from the screen and moved my hand or arm the cursor would shoot off into space...

Now again as the voice of reason having 30 years programming and user interaction work experience behind me, apple when they research/design/tested/ and prototype the iphone had to make the iphone for everybody to use, large hands small hands big ears, small ears etc ... So remember to that air mouse have had to make an app for EVERY USER, which is why the server has different options for things like speed and acceleration... Yes it does take time to zero in the mouse and yes it does take time to learn when to stop your arm moving but it's time well spent...

I think there needs to be a decent user manual that explains some of the reasoning behind some of the choices the devs have had to make to get their software out to us...

I have been informed that a new version will be released soon and that there have been many many changes to the code, so with fingers crossed there might be a better algorithm in there.

If not and you still have problems just post on the forums as I do know that the devs read these and do take note, and yes they aren't a very vocal lot but... I'd rather them be out there coding then writing 500 word explanations every 2 minutes....

Judda

[juddadredd]

Hi Jason,

Ive been messing with my iPhone 4 all morning I think your getting the gyroscope idea mixed up...
Basically a gyroscope is a 3 axis gimbal that self centres all the time, the iphone then figures out which way the iPhone is pointing in the 3 axis... So all a gyroscope does is measure orientation it cannot measure differences in height or a actual starting position for a device... It can't say I'm 2 and a half feet in the air and am 6feet away from the screen, and 3 feet from the left hand chair... If it did that then it would be super duper accurate but it doesn't, so it doesn't know exactly where it is..

And like I said the gyro can only say if the phones 39degrees up and 5degrees to the left... The gyro has been included in the iPhone to allow developers to make layers on the screen so that when you hold the iPhone up it knows which way it's pointing so as you turn around your own axis it can follow more accurately then with just the accells...

So sorry again it's just a case if trying to find the best set of speed and acc that matches the way you move your arm...

[juddadredd]

To get the program to work exactly as your require would need the iPhone to have the following A 9-axis sensor fusion algorithm (3-axis gyroscope, 3-axis accelerometer, and 3-axis digital compass) it has the 3 axis gyro and 3 axis accelerometer but only a 1 axis compass...

A better way to understand how the gyroscope works is...

A missle is fired it knows it has to go a set distance, it knows the math for a falling object and it also knows how to move it's guidance fins
Now when it checks it's gyro it then know exactly what position the missle is in, in the vertical the horizontal and it's yaw
To find out is location it would use gps
It then does the math and moves it's fins, then rechecks again
That's all the air mouse can do, it can only check V H and Yaw, it can't say I'm exactly here and pointing in exactly this direction as that would need the gps to be accurate to 1000th of a millimetre, and the accells to be so accurate that a surgical robot would look sloppy compared to it.

Hope that reads ok as it's hard sometimes to get an idea across without sounding funny unless people can hear your tone and read your face,miff anybody wants to read more on the subject invensense have a good PDF on the subject, just google it

Meantime if I can find a better algorithm for the gyro then the devs already use I will pass it along.

judda

[juddadredd]

And to put my money where my mouth is because I do like to be informed and show people that I'm not talking out of my bum...

Wiki has a good peice on Gyros http://en.wikipedia.org/wiki/Gyroscope


and here's a brief from http://invensense.com/mems/gyro/documents/whitepapers/InvenSense_MotionProcessing_MPUApps_WhitePaper.pdf

The calibration values for the compass drift easily, requiring the end user to periodically execute a series of movements to recalibrate the compass. Other compass architectures may be better, but with the penalty of a higher cost and larger size. In addition, even a good compass is limited in how well it can measure heading due to interference from magnetic isturbances in the device and in the outside environment.

Gyroscopes, also called angular rate sensors, measure how quickly an object rotates. Gyroscopes are the only inertial sensors that provide accurate, latency-free measurement of rotations without being affected by any external forces, including magnetic, gravitational or other environmental factors. This rate of rotation can be measured along any of the three axes, shown in Figure 1: X (roll), Y (pitch) and Z (yaw). Adding gyroscopes to the sensor mix will allow algorithm designers to take advantage of a pure measurement of angular velocity that cannot be delivered by compasses or accelerometers. This measurement allows for accurate pitch and roll measurements when combined with an accelerometer, and these measurements can be used to more accurately tilt-compensate the compass.

So like I said a gyroscope is ROTATIONAL DATA ONLY

[juddadredd]

And for anyone that wants to did in to the Maths behind the Gyroscope this is for you.

http://escholarship.org/uc/item/9zk2g8ph#page-1

[jason98]

.

So like I said a gyroscope is ROTATIONAL DATA ONLY

So are you trying to say that it is not possible to measure relative angular direction?
Cause like I said there is a game in app store called GunRange which exactly does what I described.
Please give it a try it and see that once you center the g-scope, no matter where you point device after that (and how fast you rotate) you can always repoint it back and the original direction will still be in synch with the scene!

That's exactly what I would expect from the gyroscope mouse!

[juddadredd]

Hi Jason,

I'm not an Air Mouse Dev I'm just another programmer whos really interested in the application as it could be the only way I enter data in to my computers in the future if its given lots of TLC :) I'm afraid its your expactations and your perspective that are a little off... that's why I posted links to actual real world documents so that you could read up on the LIMITATIONS of the iphones tech... I will try to explain the differences between Gunrange and real world use...

Now Gunrange (Been playing it all day) has a FIXED MOUSE POSITION and moves the screen display around depending on user inputs, the mouse (bullit hit) will always be centre of the screen and that's why the gyro works so well on that... right so test that... you agree on the Fixed position Good next...

Gunrange is set between four points in it's own sandboxed little world so it knows where the limits of x y -x and -y are (notice there is no Z or -z axis as thats fixed in the display routines) these are non changing constraints you can only look forwards, now that's how it maps the iPhones position to what's displayed on the actual screen by measuring the difference in the gyroscopes spin along with the Accelorometers and then centres the gun on the range... (the real world doesnt have definite positions your hand will never be in the same place in 3D) If you also look at the screen you will see that that it has a PERSPECTIVE change when you get near the ends of the range (you cant look behind you either)... You will also see that Gunrange has NO DEPTH to the game its a software trick that its in 3D, if anyone can give me a formula to figure out depth of field from the iphone to the mouse pointer on screen or projector then I'd be gobsmacked as the only way I know how to do that is with a LASER measuring device...

Now if we got the Gunrange app on a projector and started to play on that the minute we turned out of the projectors light path the game would go out of our field of view (like the mouse on the screen) and lose its position, it would still be the same first person on the actual phone but we can't see the projector...

Gunrange also has a fixed perspective (from the first person) so whereever you point the gun IS and ALWAYS will be forward which isn't the case with the mouse pointer on your screen is it?

Heres a good test for you start the game and then turn around for a bit so your body is used to the movement, dont keep your feet still actually move about the room
Now get the phone lined up just so with an object in the room (point of Reference)
Now put on a blindfold and move around EXACTLY like you did before
Take off the blindfold and are you in the same position?
Is the phone in the same position?

Well I know that both answers will be 100% NO, see your OWN INTERNAL gyroscope has limitations and is super complex as it's got your enitre brain to work on the complex computations hows that compared to the iPhone...

As I said a gryoscope is only there to make the Accelorometers more sensitive, BUT like I said earlier today I'm working on some new maths that might help out some by adding dampening and some form of calibration vs distance vs movement, but you would have to recalibrate when you did different set ups like from sitting on the sofa to standing 20 feet away from a projector...

And all I'm working with are a stream of numbers on my screen as I don't have time to create a complex user interface, but I am trying...

Hope that explaination helps

Judda

[Sixten]

I'm not sure you understand what I mean juddadredd.
There's no need for an IR receiver. The iPhone is already connected to WIFI so I am speaking only about wifi connection.
As I understand you can already use gyroscope with the Airmouse so it's already there.
Surely you don't want the mouse to go around like crazy when you are doing other things with your arms
so the trigger button is ok but maybe it could be a toggle/centering button that stays on so that you can use your thumb for left or right-click.
This should be possible but of course I cannot tell if the gyroscope is precise enough but I believe it is since it is already used in Airmouse

[MobileMouseAdmin]

That's not how any gyroscopic mice work (at least the ones we've tried). They all have a trigger button and track the relative motion to where you press the trigger. We have tried implementing what you are referring to in the past and were not happy with how it worked. Part of the problem is that you have to twist your wrist in some strange directions in order to reach parts of the screen, which can feel very natural. Another problem is that the sensitivity will vary greatly with different size monitors. Smaller monitors will be very sensitive, almost too sensitive to use, and larger monitors will be far less sensitive. We have played around with this extensively in the past but never were happy enough with it to release it. However, maybe we'll make it a setting that the user can turn on or off.


So are you trying to say that it is not possible to measure relative angular direction?
Cause like I said there is a game in app store called GunRange which exactly does what I described.
Please give it a try it and see that once you center the g-scope, no matter where you point device after that (and how fast you rotate) you can always repoint it back and the original direction will still be in synch with the scene!

That's exactly what I would expect from the gyroscope mouse!

[juddadredd]

Hi again Jason,

The Infra Red bit IS how the Wii tracks movements just reread that part as its my fingers being slower then my brain, so sorry if you got confused there.

Did you read the bumph on the gyroscope I posted as it explains what can be acheived and what can't att the moment?

[jason98]

Part of the problem is that you have to twist your wrist in some strange directions in order to reach parts of the screen, which can feel very natural. Another problem is that the sensitivity will vary greatly with different size monitors. Smaller monitors will be very sensitive, almost too sensitive to use, and larger monitors will be far less sensitive.

Well, I do not believe this because again, GunRange feels very natural and almost pixel-by-pixel sensitive. With small and big monitors one can always change the scale factor to adjust mouse sensitivity.

[juddadredd]

Jason please re-read the text I wrote above matey, You DIDN'T look at the game as I asked you to so please go look at it while Examining whats happening on screen, is it flat or 3D is the mouse fixed in the middle, is it the mouse that moves or the actual graphics on screen, youre talking about something thats completely different to whats actually reuqired to get the job done.

re-read and I'd love to hear from your persepective...

[Sixten]

Well, I do not believe this because again, GunRange feels very natural and almost pixel-by-pixel sensitive. With small and big monitors one can always change the scale factor to adjust mouse sensitivity.

Sounds great!

[jason98]

Jason please re-read the text I wrote above matey, You DIDN'T look at the game as I asked you to so please go look at it while Examining whats happening on screen, is it flat or 3D is the mouse fixed in the middle, is it the mouse that moves or the actual graphics on screen, youre talking about something thats completely different to whats actually reuqired to get the job done.

re-read and I'd love to hear from your persepective...

Judda, I don't see any problem with gyroscope mouse if it needs recalibration after I move to different angle or distance!
The whole point is to be able to lock to the screen while I seat or stand at the SAME location and that's what GunRange does very well. Now I am not sure why are stressing on the fact that the game rotates the scene around me while gyroscope mouse does not. I do not really see any problem here again. If GunRange can translate gyroscope input into x and y scene rotation angle why would gyroscope mouse not be able to do the same with the pointer? All it needs is to convert x/y angles to x/y position using some scale factor that depends on sensitivity calibration. Again it's ok to have the user to recalibrate once she moves. And the calibration input can be just asking her to point device to the left bottom corner of the screen and then right top one.

[juddadredd]

Hi Jason,

Hope your well I was looking forward to talking with you again as I spent the night ripping apart game code just for you my friend. Righty oh I can say 100% that the gyroscope in Gun Range isnt converting x/y angles in to a realative position thats magnified by the user moving there arm and hand, its moving a graphics render by the amount of change in the axis pixel for pixel, I know that as I had its code all disassembled and all warm in my hands last night so don't even think about saying anything different as you will loose more face then you already have...

Now you still didnt answer any of my questions just because you know that the answers you will give go hand in hand with exactly what I've said and your just being what I call pig headed, so if I can't get it over to you in my words then I'm going to use your own words against you!!! Sorry but you dug your own one on this and I'm a little fed up with your attitude...

You have it EXACTLY RIGHT at last that by calibrating xy to +x+y (bottom left to top right) that will help you get a better bead on the cursor not 100% precise but better, and you have hit the nail EXACTLY on the head with this comment these are YOUR own words "Convert x/y angles to x/y position using some scale factor that depends on sensitivity calibration"

So I'm going to say "server application settings", Pointer Speed and Acceleration are they or are they not in threre? yes they are
Have the developers given you the EXACT Tools to calibrate the iPhone to match the action you just described? yes they have

I use air mouse on my iPhone 4 about and hour a day as I mostly spend time in meetings now in combination with my iPads mobile version it works even better as I have a seperate keyboard to use. I have never ever had a single problem with the air mouse, of course there are things I wish it did or things I wish it did better but its a work in develoment, but then again my hand eye co-ordinations quite good after 30 years of playing games.

And to finish this off again in your exact words "As far as I understand, with a gyroscope whenever I point the device into the original direction I expect the mouse pointer to return to the original position where the gyroscope was engaged." Which I'm going to point you back the articles that I spent my time searching for so that you UNDERSTOOD what a gyroscope actually did and that by YOUR UNDERSTANDING of what it actually does might change YOUR EXPECATIONS to meet real life...

Your own words not mine...

[jason98]

Hi Jason,
So I'm going to say "server application settings", Pointer Speed and Acceleration are they or are they not in threre? yes they are
Have the developers given you the EXACT Tools to calibrate the iPhone to match the action you just described? yes they have


You must be joking, right? You think I posted multiple messages here just to hear that all I am asking is already supported?

[juddadredd]

Duuudddeeeeee! your just so funny, need an ambulance as i think your going to stroke... and you still didn't answer my questions by the way!!!

In all seriousness I need you to try an experiment for me ... Remember I'm not a dev I'm just someone who's more educated then you are in gyros and coding and tyring to help out...

Get your screen res in to exactly 1280 x 1024
Now find an area of wall and stand exactly 4 foot away not more not less as I'm going to put that in to my model.
and again exactly like I say get 4 A4 sheets of paper and tape them against the wall so that it matches up with your screen res (see what I'm doing yet?)
right now we should be working on EXACTLY the same measurements (hopefully)
Now calibrate the mouse to go from the bottom right to the top left and top right to bottom left in the WINDOW not the paper, you using the paper as a guide as its centre will be 0,0 0,0 and I'm going to try to work from there OK..

this should be exactly as I've got on my screen and they should match up

Now roughly how far off of the A4 paper does the phone move?
in which direction does it leave the area.

[Sixten]

Come on juddadredd please keep a gentle tone here. It is not necessary to be aggressive to get your points through.
I got my iPhone 4 and it's amazing. I find it a lot easier to use Airmouse now with gyroscope control.
I don't wanna get into those experimets you describe there because it's not really necessary.
I agree with Jason since I can test that gyroscope control really works and it shouldn't be that difficult to lock it to the screen
and using a recalibrate button now and then.
I notice there's some drifting of the mouse pointer when I press the trigger button and move it a little. Hopefully that can be fixed.

[juddadredd]

Hi six,

I'd I had already pm'd Jason when i wrote the above to let him know I was pulling his leg, just reread the above as a joke...

Look if you guys won't help me by finding out where then drift is and by how much over a set distance then your now on your own as my version is 90% accurate and that's enough for me so overall I'm very very happy...

For the record An air mouse doesn't centre to the screen not a single one of the ones I've used do that, all they do is move a mouse pointer remotely... A gyroscope if either of you had bothered to read the links doesn't centre anything in the physical world, it can't do it won't do but as you guys wanted it I tried to help the devs achieve that goal, by writing some algorithms that mimic a 3 axis compass and have just finished the dampening code...

So far I've spent more then the total of 20 hours of my free unpaid time on this request to date, have either of you got any idea on the maths behind something like that?. And now I can't be arsed with it... So both of can help out and I will use the version that you guys come up with and you know what it better centre...

[jason98]

I notice there's some drifting of the mouse pointer when I press the trigger button and move it a little. Hopefully that can be fixed.

Exactly. And it is really annoying. Also hit and edge of the screen and keep moving beyond then go back. See the problem?

[juddadredd]

Instead of just moaning about it do the test and then get back to the devs with some hard data but oh no that's too bloody hard for you both to put yourselves out isn't it?

[Rayman2170]

Exactly. And it is really annoying. Also hit and edge of the screen and keep moving beyond then go back. See the problem?

Seriously dude? Ok sit at your computer, grab hold of the mouse and move it. Keep going till the cursor hits the edge of the screen, and then continue moving the mouse even farther beyond that point. Now start moving the mouse back in the other direction. Gasp! It starts moving immediately! It's a "limitation" of mice (exactly the type of device this app is emulating), get over it. A mouse is strictly an input device; it has no idea where the cursor is on screen.

If you want precise mapping of pointer position, either get a tablet or you'd need some kind of IR device similar to Wii.

P.S. Sorry for resurecting an old thread, I just had to respond to this. lol

[juddadredd]

Cheers rayman I've not laughed so hard in weeks, your my hero today... Shame the devs never took the time to back me up like you have just done, so cheers for that one...

I even made them a calibration screen so that they could train themselves to use the kit be to no avail, and when you ask them to help with some Imperial data the twats don't answer again... Lmao