[feature]: persist payment+invoice HTLC custom wire records and return them in RPCs

[lukegao209]

[Maintainer edit: Depends on lightningnetwork/lnd/issues/8769]
[Maintainer edit]: TODO

• extend itests in litd to ensure this property holds

Is your feature request related to a problem? Please describe.
When using stable-coins to pay a regular BTC invoice, the user is effectively selling stable-coins and buying BTC.
However, the current sendPayment interface does not return the amount of stable-coins sold, making it difficult for the Edge Node to maintain its balance and perform reconciliation. This is nearly the final step required for the Edge Node to operate normally.
hope this can be addressed with high priority by:

1. Having the sendPayment API return SendPaymentResponse_AcceptedSellOrder, and
2. Providing an query api to retrieve historical information

type SendPaymentResponse struct {
	state         protoimpl.MessageState
	sizeCache     protoimpl.SizeCache
	unknownFields protoimpl.UnknownFields

	// Types that are assignable to Result:
	//
	//	*SendPaymentResponse_AcceptedSellOrder
	//	*SendPaymentResponse_PaymentResult
	Result isSendPaymentResponse_Result `protobuf_oneof:"result"`
}

for {
		select {
		case <-ctx.Done():
			return ctx.Err()

		case <-r.quit:
			return fmt.Errorf("server shutting down")

		default:
		}

		update, err := updateStream.Recv()
		if err != nil {
			return err
		}

		err = stream.Send(&tchrpc.SendPaymentResponse{
			Result: &tchrpc.SendPaymentResponse_PaymentResult{
				PaymentResult: update,
			},
		})
		if err != nil {
			return err
		}
	}

[guggero]

Thanks for the feature request.
I'm not sure I understand the first part of it. We already do send the SendPaymentResponse_AcceptedSellOrder here:

taproot-assets/rpcserver.go

Line 6903 in 139669b

err = stream.Send(&tchrpc.SendPaymentResponse{

That is only sent when paying an invoice though, not when using keysend.
Do you not receive that message?

For the second part, I'm going to extend the lnd RPCs with the custom data fields that we also have for ListChannels, PendingChannels and ChannelBalance.

Adding the custom data to the lnrpc.InvoiceHtlc and lnrpc.HtlcAttempt should be enough for you to find out how many assets you received for an invoice or how many you paid for a payment.
Does that sound good?

[lukegao209]

For the first part, it seems that the assets amount is not being returned

[lukegao209]

Adding the custom data to the lnrpc.InvoiceHtlc and lnrpc.HtlcAttempt should be enough for you to find out how many assets you received for an invoice or how many you paid for a payment.

For the second part, I agree with you; it seems like a great idea.

[guggero]

But you do get the SendPaymentResponse_AcceptedSellOrder response, right?
Then you can calculate the number of assets exactly like this:

taproot-assets/rpcserver.go

Line 6913 in 139669b

numUnits := uint64(*invoice.MilliSat) / msatPerUnit

[guggero]

I looked into the second part ("Providing an query api to retrieve historical information") a bit more and it looks like that's not going to be super quick. We don't currently persist the wire-level custom message data that's attached to the HTLCs in the invoice and payments. We need to add that in lnd first, see some of the open issues here: lightningnetwork/lnd#8769

So as a workaround you unfortunately need to store the buy or sell quote from the AddInvoice or SendPayment calls so you can calculate the number of units as mentioned above.

We'll look into fixing this soon, but can't promise anything as it's quite a bit of work required still.

[lukegao209]

I looked into the second part ("Providing an query api to retrieve historical information") a bit more and it looks like that's not going to be super quick. We don't currently persist the wire-level custom message data that's attached to the HTLCs in the invoice and payments. We need to add that in lnd first, see some of the open issues here: lightningnetwork/lnd#8769

So as a workaround you unfortunately need to store the buy or sell quote from the AddInvoice or SendPayment calls so you can calculate the number of units as mentioned above.

We'll look into fixing this soon, but can't promise anything as it's quite a bit of work required still.

understand. thanks

[dstadulis]

Part 3 and 5 of staging branch will ship this

[guggero]

We'll want to update the litd integration tests to assert these changes though, so won't be exclusively fixed by the mega staging branch being fixed.