solana: support tokens with transfer hooks

[kcsongor]

This PR adds support for token2022 tokens with transfer hooks. For more details on transfer hooks, see https://solana.com/developers/guides/token-extensions/transfer-hook.

At a high level, a transfer hook is a program that is called (by the token program) each time a transfer (of a token of a specific mint) is made. The key difficulty is that the transfer hook program may require additional accounts, which need to be propagated through the instruction.

This PR modifies the on-chain code to call the appropriate helper function that propagates the necessary accounts, and also the off-chain code (ts sdk) that derives these account addresses based on the on-chain schema standard that token2022 introduces for transfer hooks. The rust tests (cargo test-sbf) still test against the legacy spl token, while in this PR I modified the typescript tests to test against a token with a transfer hook. This way, both paths are tested appropriately.

There is a dummy transfer hook in the test that bumps a counter on each transfer, and has a dummy account seeded by the owner field of the sender token account, which is just used to test that the account information is properly propagated.

Adding this was the motivation for #391, because the on-chain helper code is simply completely unusable (broken) in earlier sdk versions.

ABI changes

Breaking changes

This change is breaking, so it warrants a major ABI version upgrade (bumped the version to 2.0.0). This is because we now have to perform an extra transfer when burning and minting tokens in order to trigger any transfer hooks. This extra transfer in turn requires some additional accounts -- refer to the documentation in the transfer_burn and release_inbound_mint functions for more details.

New features

Also added an extra set of instructions to programmatically create/manage address lookup tables. These were necessary to add extra transaction size headroom for transfer hooks to be triggered. Without a lookup table, a transfer hook could use up to 2 custom accounts, but now there is space for 15+. See the documentation in lut.rs for more details.

[johnsaigle]

No opposition to the changes so far. Looks like this needs a bit of work to get past CI

[kcsongor]

Hm, interesting. anchor test passes for me locally, I can't reproduce the CI failure here (which seems to be deterministic) :(

[johnsaigle]

@kcsongor I'm getting the same issue as CI when I run make anchor-test locally.

Maybe check your local CLI tool versions? Here are mine:

avm 0.29.0
solana-cli 1.18.10 (src:50b3f337; feat:3469865029, client:SolanaLabs)

[kcsongor]

avm 0.29.0
solana-cli 1.18.10 (src:50b3f337; feat:3469865029, client:SolanaLabs)
anchor-cli 0.30.0

for me

[johnsaigle]

I have anchor-cli 0.29.0. Maybe you could try downgrading?

[kcsongor]

Downgraded to 0.29.0, it still works. But I managed to reproduce this on my x86 ubuntu machine (it's working on arm64/macOS). What arch/OS are you running on?

[johnsaigle]

I'm using Darwin arm64. Strange that we're getting different results.

[johnsaigle]

I haven't done an in-depth review yet but I've assigned myself and will return to it later today or this week.

Quick note: Are the changes to the Solidity files meant to be here or are they the result of the rebase? I'm unclear how they're related to the changes for the Solana code.

[kcsongor]

Ah that's right -- the base branch of this PR is #399. I rebased this one, but not that one yet, so github shows a larger diff. Will fix that by rebasing #399 also.

[kcsongor]

Done, the diff shows up correctly now -- thanks for pointing it out!

[johnsaigle]

A few questions about the API. Will approve after we discuss.

[kcsongor]

Added a new commit which makes some non-trivial changes. tldr we now perform a transfer before burning on the way out, and a transfer after minting on the way in. This is done in order to trigger the transfer hooks of the underlying token, which minting and burning don't trigger.

CI fails because now there are too many accounts in the transfer transaction (since it does an approve, then a transfer, then a release outbound via wormhole. The transfer hook adds 3 extra accounts, which just about tips it over the limit. I'll have to add a lookup table to the sdk).

outdated

[johnsaigle]

Added some notes, mostly to confirm that removing some of the checks should be safe. I like the Anchor constraint changes for the Mode checks and the comment additions quite a lot.

[johnsaigle]

Looks good on my side from a security perspective. Csongor and I discussed some ideas on Slack and I'm comfortable with the state of the PR.

[nonergodic]

no additional changes since my last review from what I can tell

the only thing that stuck out to me when taking a second glance is that the ntt quoter version also got bumped to version 2.0 even though for the ntt quoter itself nothing actually changed, which might be slightly confusing ("What's the difference between ntt-quoter 1.x and 2.0?" "No difference.") though ofc not bumping it is maybe also bad ("Is this the right version of the quoter for that version of NTT?").