References Transactions would work fine to charge a separate transaction for the tip, but then the customers would see two separate charges on their credit card and you'll probably get lots of questions and confused people wondering why their card was hit twice. As obvious as it seems, I've been there.
The best way to handle it would be to run an Authorization when you first run the card. Authorize it for something like 50% above the ticket total, or whatever you think it would take to cover the final total including a tip for that order.
If the order is canceled for any reason you simply Void the authorization. If you're ready to actually process the funds with a final total including tip, then you can Capture the amount that you need up to the amount that you originally authorized. I believe you may even be able to capture up to 10% more than the original authorization, but I'm not 100% sure on that.
The thing you would need to be careful about is authorizing a card for some large amount and then not voiding or capturing shortly after. The authorization is only guaranteed for 3 days. You can run a re-authorization and get an additional 3 days if you need to.
If the order gets canceled and you don't void it then you'll get complaints that a large amount is "charged" on their account. It looks like a charge until it clears back or is voided. If you don't void it then it can take up to 29 days for them to get access to those funds on their account again, so you want to avoid those types of complaints too.
You would need Payments Pro to handle credit cards directly for that sort of a system. Depending on which version of Pro they put you on you'll use either the PayFlow API, or you'll use variations of DoDirectPayment, DoVoid, DoCapture, DoReauthorization.
