Multitouch Game Loop Manipulation Processing

By
Dave
Project
Published
12 Aug 2010 23:05
Last Modified
8 Dec 2012 16:21

Multitouch Surface support for XNA-based applications is provided via the Microsoft.Surface.Core.Manipulations namespace in an event-driven model. I was interested in investigating general multitouch processing within a game-loop model, i.e. not involving events, and thought it would be instructuve to develop my own class to process manipulations in this way.

In a similar manner to the Affine2DManipulationProcessor, I wanted to support arbitrary combinations of translation, rotation, and scale. Translation is trivial, since it is simply a matter of tracking the changes in each manipulation point between calls and taking the average.

Rotation and scale is a little more tricky, since each of these values are relative to a centre-point, or manipulation origin. Rotation is calculated from the average angle between each point and the origin, and scale is calculated from the average distance. Unlike a mouse cursor, manipulation points come and go, so the key is only to track changes to those points which persist between calls, and then update the positions, origin, distance and angles for the next call.

In order to generalise manipulation points from multiple sources such as mouse, gamepad, Surface Contacts and .NET 4.0 Touch, I created my own Manipulation structure to abstract properties such as ID and position. My input processing can then build a collection of these objects from any relevant source(s) and pass them to my manipulation processor as part of the Update call as follows:

myManipulationProcessor.ProcessManipulators(myManipulators);

The cumulative transforms for translation, rotation and scale (according to the supported manipulation types specified) are then immediately available as properties on the manipulation processor.

In order to test the solution, I wrote a small harness to visualise the manipulation points, and their effect on a reference cross-hair. While I also added support for a mouse (right-click to add/remove points, left-drag to move points), multi-touch hardware is required to test scenarios when multiple points are added, moved, or removed simultaneusly. A screenshot is shown in Figure 1.

Multitouch manipulation

Figure 1. Multitouch manipulation with averaged manipulation origin.

One of the complexities of working with multiple manipulation points is deciding how to determine the manipulation origin. One option is to simply take the average position of each point, as shown in Figure 1. Another option is to introduce a speicific "pivot point" as the origin and use this to calculate scale and rotation. This pivot point can either be fixed, or tracking the object being manipulated. The latter two options are shown in Figures 2 and 3.

Multitouch manipulation

Figure 2. Multitouch manipulation with fixed pivot point.

Multitouch manipulation

Figure 3. Multitouch manipulation with pivot point centered on manipulation object.

The solution is demonstrated in the following video. Each approach for determining the manipulation origin is illustrated using translation, rotation, and scaling manipulations, initially in isolation and then in combination.

Video 1. Multitouch game loop manipulation processing.

Comments

By
KoichiSenada
21 Jan 2011 11:52
Hello, Dave, thank you for the great article!
How do you calculate the average angle so that it doesn't glitch while crossing the zero value?
Please reply, I just can not find the algorythm anywhere.
By
Dave
21 Jan 2011 17:19
Hello. If you use vectors, you shouldn't need to worry about wrapping angles. I'm tracking the average change in angle for each manipulator between each frame. In order to calculate the angle between two vectors, if 'from' is the vector from the origin to the manipulator for the previous frame, and 'to' is the vector for the current frame, one way to calculate the angle is:

angle = (float)(Math.Atan2(to.Y, to.X) - Math.Atan2(from.Y, from.X));

If you need to wrap angles in XNA, use:

Microsoft.Xna.Framework.MathHelper.WrapAngle(angle);
By
Darkknight
6 Dec 2012 06:35
Hello, Dave
Great article. The Atan2 function returns the angle in radians. So, when you compute the average in change for all fingers one of the finger will reach the end of a circle whether it is a 0, 360, 180 or -180 in that moment the object you are rotating will jump because the delta of change of that finger will not be something like "0.112321" but it will be in degrees "359" and that will flip the object instead of just rotating it. so how did you over come this i need a sample code if you can? Please.
By
Dave
7 Dec 2012 00:38
Hello. The Math.Atan function returns a value between -180 and 180, so the key is to "wrap" the delta between previous and current angles to between -180 and 180.

For example, if we "cross" the boundary from 1 to 359deg, the angles are 1 to (359 - 360 = -1), so the delta is 1 - -1 = 2deg. Similarly, 179 to 181deg is 179 to (181 - 360 = -179), the delta is 179 - -179 = 358, which wraps to 358 - 360 = -2deg.

If you are using XNA, use:

Microsoft.Xna.Framework.MathHelper.WrapAngle(angle);

If not, the pseudocode is:

if (angle > 180)
angle = angle - 360
else if (angle < -180)
angle += 360

Regards,
Dave
By
Darkknight
8 Dec 2012 07:23
I am sorry Dave but still doesn't work as i want it. Here is my code may be you can tell me what am i doing wrong?

I am using C++ builder XE3:

float __fastcall TForm1::AngelFromPoints(TPointF p1, TPointF p2)
{
float angle = 0;

if (p1 != p2)
{
angle = RadToDeg(atan2((p1.Y - p2.Y),(p1.X - p2.X)));
}
return angle;
}
//---------------------------------------------------------------------------
void __fastcall TForm1::ComputeRotaionDelta()
{
TotalDelta = 0;
TPointF Total = PointF(0,0);
Memo1->Lines->Clear();

for (int i = 0; i < 5; i++)
{
Total += Points[i]->Position->Point;

AbsCen = Hand->LocalToAbsolute(Center);
Angles[i] = AngelFromPoints(AbsCen, Hand->LocalToAbsolute(Points[i]->Position->Point));

if (Angles[i] > 180.0f) Angles[i] -= 360.0f;
if (Angles[i] < -180.0f) Angles[i] += 360.0f;

TotalDelta += Angles[i] - curAngles[i];
Memo1->Lines->Add(String(Angles[i] - curAngles[i]));
curAngles[i] = Angles[i];
}

Center.X = Total.X / 5.0;
Center.Y = Total.Y / 5.0;

rotatdelta = TotalDelta / 5.0;

CenterPoint->Position->X = Center.X;
CenterPoint->Position->Y = Center.Y;
}
//---------------------------------------------------------------------------
void __fastcall TForm1::TrackBar1Change(TObject *Sender)
{
Hand->RotationAngle = TrackBar1->Value;
ComputeRotaionDelta();
Target->RotationAngle += rotatdelta;
}
//---------------------------------------------------------------------------

I am using five points to rotate another rectangl object "Target". And hand is another rect that contains the five points, when i rotate the hand rect the 5 points also get rotated with it and compute angles from the absolute coord of every point and the center. So, any ideas?
By
Dave
8 Dec 2012 12:31
My first thought would be to look at the line:

TotalDelta += Angles[i] - curAngles[i];

Even though Angles[i] and curAngles[i] are each wrapped to between -180 and 180 degrees, the difference between them is not. Try the following, where WrapAngle(angle) uses the pseudocode above:

TotalDelta += WrapAngle(Angles[i] - curAngles[i]);

So if a point moves from 179 to -179deg, the difference is 179 - -179 = 358deg, which wraps to 358 - 360 = -2deg. You therefore only add -2 to your total, rather than 358.
By
Darkknight
9 Dec 2012 15:35
Thanks that worked really well.
Now for scaling I believe it’s the same idea:
1- Compute the distance between every touch point (let's say 5 points) and the center point.
2- Take the average for the 5 distances.
3- Now what how can I extract the scale delta. Because scale delta is different from rotation and translation it’s not an additive value like the others. It's ok for me to get the scale change between the current and previous frame.
But I saw an example says that the scale delta can be computed like this: scaledelta = 1 / averageDistance; without computing any changes.
So, can this work too or do I have to compute change in values.


Add Comment

*
*
*
Captcha
*
*Required