Ordered consumers don't have all of the same settings as unordered ones

[paolobarbolini]

Proposed change

Give pull::OrderedConfig most of the options as pull::Config, instead of exposing just a few of them and hard-coding the rest like it's currently being done:

nats.rs/async-nats/src/jetstream/consumer/pull.rs

Lines 695 to 729 in 1303b8b

	impl IntoConsumerConfig for OrderedConfig {
	fn into_consumer_config(self) -> super::Config {
	jetstream::consumer::Config {
	deliver_subject: None,
	durable_name: None,
	name: self.name,
	description: self.description,
	deliver_group: None,
	deliver_policy: self.deliver_policy,
	ack_policy: AckPolicy::None,
	ack_wait: Duration::default(),
	max_deliver: 1,
	filter_subject: self.filter_subject,
	#[cfg(feature = "server_2_10")]
	filter_subjects: self.filter_subjects,
	replay_policy: self.replay_policy,
	rate_limit: self.rate_limit,
	sample_frequency: self.sample_frequency,
	max_waiting: self.max_waiting,
	max_ack_pending: 0,
	headers_only: self.headers_only,
	flow_control: false,
	idle_heartbeat: Duration::default(),
	max_batch: 0,
	max_bytes: 0,
	max_expires: Duration::default(),
	inactive_threshold: Duration::from_secs(30),
	num_replicas: 1,
	memory_storage: true,
	#[cfg(feature = "server_2_10")]
	metadata: self.metadata,
	backoff: Vec::new(),
	}
	}
	}

Use case

Ingest all messages with batching acknowledgement (AckPolicy::All) or guarantee ordered ingestion of messages while keeping acknowledgments.

Contribution

Yes

[Jarema]

Hey!

Ordered consumer is a client side construct, and most of the settings are required for proper functioning.
The only reason why normal pull consumer can get messages out of order are redeliveries. Those would happen if you have max ack pending > 1.

So Ordered Consumer has ack: none and max deliver = 1, which ensure that redeliveries will never happen, while still having very high performance characteristics.
This is achieved by having in memory, r1 consumer.
The consumer ensures no messages lost without acks by recreating itself by the client whenever a missing sequence is detected.

Because of all above, those settings are locked how they are.
Enabling ack would make this whole idea no longer work.

Did you consider having a normal pull consumer with max ack pending = 1?
That way you would have both acks and ordered guarantees.

[paolobarbolini]

The part about being client side makes perfect sense, and that's what I was expecting too. Everything else though seems not to work the way I was expecting it to.

Looks like I'll have to write myself a wrapper that does At Least Once delivery (re-deliveries are fine) using batching, making sure to process messages in order by detecting dropped messages + re-deliveries already being handled on the application side.

[Jarema]

If you have max_ack_pending set to 1, you will get guarantee of order, though without being able to ack many messages at once.

Otherwise, you can leverage the message metadata to check deliver count for redeliveries/dropped messages with your own logic. However, if performance is not critical (as max_ack_pending = 1 will be a bit slower as it delivers next message after ack for previous one was received), that is much simpler solution without any custom code on your end.