carry this connection ID for the duration of the connection or until its peer invalidates the connection ID via a RETIRE_CONNECTION_ID frame (frame-retire-connection-id).  Connection IDs that are issued <del> and not retired are considered active; any active connection ID can be used. </del>  An endpoint SHOULD ensure that its peer has a sufficient number of available and unused connection IDs.  Endpoints store received

Note also the editorial changes in , which are great and still needed: this adds a little bit more to the reference from those changes to do the key clarifying for . The remaining clarification about is coming in the SPA PRs, but I wanted to look at this paragraph in one place.
Section 12.2 makes two quasi-conflicting statements about coalesced packets. In a connection where the sender has multiple CIDs in hand, using two different CIDs in the two coalesced packets is entirely consistent with the guidance to the sender, but would be dropped based on the guidance to the receiver. We should align these two statements.
Fixing this would require a change to normative language. I'm of the opinion that the first should change to different connection IDs rather than connections; the second might change to a MUST NOT, but it could remain a SHOULD and achieve the same end. The main reason for this is that routing will be based on the first packet. But there is also a secondary benefit in not creating a strong correlation between different connection IDs for the same connection.
But this complicates the coalescing logic, because you no longer only need to look at packet types but now also CID fields. At least in my stack, that is an ugly change to make at this stage.
The alternative, which I can now see is probably a smaller change is:
What motivated this was two observations: we don't coalesce except during the handshake, where we are not migrating some stacks generate packets for coalescing while processing, and so they might generate a datagram with multiple connection IDs
NAME writes: This is much simpler alternative than what is being proposed. Now an implementation can only drop a packet after doing a connection lookup, whereas before it could just compare the CID to the previous value. If this is just a matter of personal preference, I vote for this approach, so now the score is 1:1 (with NAME being on the opposite side). To take a step back: why is recommendation to drop packets exist in the first place? If a sender wants to coalesce packets from different connections, why not let it?
I think it should not allow having multiple packets for different connections in a single UDP datagram - please. Currently I see implementations (including mine) have something like: Receive UDP packet. Parse packet header. Check CID, find the connection handler. Pass the packet to the respective connection (thread) to handle the packet. I think it makes implementations much more complicated (already). Also I wonder it may cause security issue if having bad assumption?
The classic example would be coalescing Handshake packets, whose DCID is the SCID used by the server, and Initial or Zero-RTT packets, whose DCID is the client-chosen Initial DCID. The client MAY switch to using the server chosen CID for Initial packets. Maybe we should say that it MUST do that if it is going to coalesce Initial and Handshake packets. Not so clear about 0-RTT packets. Is it actually allowed to send them with a different DCID than the Initial DCID?
NAME : Our assumption has been that load balancers, for instance, can simply route on the CID in the first header they see in the UDP datagram. If we allow different connections to be coalesced, that complicates things. It may also start to leak some things about the way load is distributed within the server network: if some CIDs when combined cause loss and some don't, that reveals some things about how the server load is distributed. Is there a strong argument for changing this behavior? This new language in was always the intent. I suspect it didn't change when we started allowing multiple CIDs within a connection. Unless there's an argument for changing behavior, this is really an editorial PR.
NAME If I understand correctly, every time you change the connection ID, you change the connection ID for all packet number spaces. Assuming that my reading of the spec is correct in this regard, it seems like there's no compelling reason for making this change?
NAME the spec actually says (7.2): The first flight of 0-RTT packets use the same Destination Connection ID and Source Connection ID values as the client's first Initial packet. Upon first receiving an Initial or Retry packet from the server, the client uses the Source Connection ID supplied by the server as the Destination Connection ID for subsequent packets, including any 0-RTT packets. SO you are correct, there is no ambiguity.
NAME Thank you for looking this up. Then I'm opposed to making any change here. Not only is it unnecessary (that alone should be enough), it would also make it more difficult to evolve the way we deal with coalesced packets in the future: I can imagine a future QUIC version introducing a way to authenticate the entire coalesced packet, while at the same time compressing the header. This will be easier if we keep the explicit property that all connection IDs of the coalesced packets match.
NAME I'm not sure if I agree. Coalescing of QUIC packets is a feature specific to v1, entities that operate without the knowledge of the connection (e.g., load balancers, datagram dispatchers) are not expected to consult the CID values of QUIC packets that are appended. Therefore, what we are talking here does not affect what we can do in the future.
That argument was based on the thought that it will be easier in a future QUIC version because the diff to v1 would be smaller. My main point is that this change is unnecessary because when endpoints change connection IDs, they change it for all packer types at the same time.
NAME Thank you for the clarification. I think I do not agree to this, too. IIUC, the consensus we reached with was that the CID of a Handshake packet and that of the 1-RTT packet can be different. While it is true that the initial DCID set by the client is expected to change when the client receives the first "active" connection ID supplied by the server, the fact does not mean that the CIDs of all QUIC packets within a datagram has to be the same in all the cases.
I think we need to make a change, in one direction or the other. It's unfortunate to mix CIDs, since it clearly links them to any observer, but doesn't seem like a big deal. For some implementations, it sounds like it's a big deal to avoid mixing them. Therefore, while I'm content with either resolution, it seems like it makes more sense to resolve toward being more permissive; e.g. . Mixing packets from different connections seems like a generally bad idea, since it assumes an implementation structure on the far end for it to work. The more likely outcome is that coalesced packets from other connections will simply be dropped as invalid. (Of course, this very property could be used for padding / misleading observers.)
Hi Mike, Can you say more about why you don't see this as an issue? Is it because this use case is most likely during the handshake? Or is there some other linkability already present that makes this less of an issue? regards, Ted
Some implementations try to change the CID as soon as possible, to avoid linking 1RTT traffic with the CID visible during the handshake and to improve privacy. There will be a brief window when they still need to send both some handshake packets and some 1RTT packets. But coalescing a Handshake packet with the old CID and a 1RTT packet with the new CID links old and new CID and thus negates the effort. And just in case someone asks, no, you don't really need to decrypt the packets to access the second CID. You just need the coalesced packet and another 1RTT packet. Not hard.
NAME If that is the concern, wouldn't it be better for endpoints to migrate to a new CID after learning that the peer has dropped the Handshake keys? If an endpoint switches to the new CID before the handshake keys are dropped by the peer, there's chance that the peer might use a new CID in the Handshake packets that the peer sends. That at least leaks when the connection was established for that CID.
NAME yes, the simplest way to not run into coalescing problems is to not start migrations or CID changes before Handshake Done.
Apologies, I'm getting lost in the weeds here. I agree with NAME that we need to make a change, and that is really meant to fix the text to what the intent always was. NAME NAME NAME Are we all agreeing on this point?
NAME That is my view. I think to be an editorial amendment of . Regarding linkability problem, the simple and safe way of mitigating the problem (as proposed in URL) does not interfere with the proposed resolution.
NAME I personally prefer the old version, force single CID in coalesced packet. Loosening the text to allow multiple CID requires a code change in my implementation.
I'm with NAME Single CID.
Both, actually. This occurs during the handshake, when addresses are required to be stable. Unless you're running a large number of parallel connections, all CIDs used on that 4-tuple during the handshake belong to the same connection with very high probability. You can break that linkage by deliberately changing CID and port at the same time, but you can't do that until the handshake is confirmed. If you want the linkage broken, you need to do that jump post-handshake anyway.
Hi Mike, I think "high probability" is probably app-dependent, because there may be cases where there are significant numbers of parallel connections. In the case of proxied queries, for example, the use of the same 4-tuple but a different CID could turn out to be common, because the proxy may prefer to keep traffic from its different clients separate even if they could be served by the same upstream server. A DNS example: if you have an DNS resolver talking to an upstream proxy or a common authoritative server, that resolver may want to set a different Client Subnet for different query streams, so that input can be used to determine the preferred reply. In the presence of a NAT or CGNAT, there could be interesting interaction with the dwell timers in the state table of the middlebox. If you change it too late, the middlebox may recycle the most recently freed port, which could result in you getting the same 4-tuple to an observer on that part of the path. Speaking strictly as an individual, I think using a single CID per coalesced packet as Christian prefers is cleaner and leaves open app behaviors we may not have yet identified. regards Ted I'm not a big fan of Client Subnet, and these particular parallel connections may or may not pose a risk; it's just a proxy situation where the parallelism might occur.
The discussion is active, I'm marking this as design.
NAME I might be missing something, but if an endpoint wants to cut linkability between the packets used during the handshake and the packets used in 1-RTT, that endpoint has to switch to a new CID after the peer has dropped its Initial and Handshake keys, regardless of the outcome of this issue. That is because if the endpoint switches to a new CID while the peer is still sending Initial or Handshake packets, we allow the peer to use that new CID in the long header packets sent in response. Therefore, I do not think that requiring endpoints to use one DCID in all the coalesced QUIC packets help us in practice.
In my implementation, ngtcp2, it processes packet in the way that PR proposes; which means I just over looked existing text, but anyway, I think that processing a packet atomically is cleaner approach in implementation wise rather than passing a some kind of pointer to the very first packet header.
NAME Once a non-coalesced packet is passed to a connection, in my implementation I don't have to do any checking. The server already makes sure to pass packets to the correct connection based on the DCID. So there's additional validation logic required for coalesced packets. And the easiest way is to just check that all DCIDs match. Sure, it would be possible to somehow query which other DCIDs would also be valid for the same session, but it's more complicated. And given that nobody has come up with any good reason to coalesce packets with different DCIDs in the first place, I'm wondering how this additional complexitiy would be justified.
I don't foresee a large problem with the proposed change, but I also don't think it's necessary.
Do we have a proposed resolution for this one?
I think we settled on requiring connection IDs to be the same, but I haven't had the time to write that up, so not just yet.
NAME FTR, discussion happened on the mailing list . My sense is that it's still not clear to some (including me) why the current fix in (requiring "connections" to be the same) doesn't work, but nobody is arguing strongly against what NAME said above (requiring "connection IDs" to be the same). So that might be the least common denominator.
added the text " Connection IDs that are issued and not retired are considered active; any active connection ID is valid for use at any time, in any packet type. " As far as I can tell there is no text specifying that an issued connection ID cannot be used with subsequent connections. The introduction to connection IDs state that there must not be correlation across connections, but if an issued connection ID was never consumed, there would be no correlation because it was transmitted encrypted.
We're currently not specifying which DCID is supposed to be used on retransmissions of Handshake packets. A client might receive a NEWCONNECTIONID frame in the first couple of 1-RTT packets it receives, before receiving either a HANDSHAKEDONE or an acknowledgement for a 1-RTT packet is sent itself. Therefore, it's still running loss recovery on the Handshake packets it sent out earlier. If there's an acknowledgement outstanding for one of those Handshake packets, the client will have to retransmit that packet. Which DCID is it supposed to use on the retransmission? Is it the DCID that was used on the original packet? Or is it the CID provided in the NEWCONNECTIONID frame? Note that the NEWCONNECTION_ID frame might already have requested the retiring of the CID used during the Handshake.
As I can see, there are three options: a) let the receiver of NCID to send Handshake packets using the new CIDs b) require (or recommend) receiver of NCID to continue using the original DCID until the handshake is confirmed c) require (or recommend) the sender to withhold the emission NCID (with a non-zero RTP) to until it sees an ACK for handshake confirmation I prefer (c) (or maybe (b)), because (a) has corner cases even though it might seem easy. Consider the case where a server sends TP.preferredaddress then as the first packet, sends the NCID frame with RTP set to 2. If we adopt (a), this would mean that the client would have only one CID that it can use (the one being embedded in the NCID frame) even though it needs two (one for the original path, and another for the path specified by TP.preferredaddress). As you can see, the design of the NCID has been based on the assumption that providing one new CID alongside a retirement request guarantees that the receiver would have enough active CIDs. But in this particular case, that is not the case. Based on this observation, and based on my assumption that a non-zero RTP would be sent by only some of the endpoints (the necessity of CID retirement is questionable for HTTP/3, the scope of QUIC v1, as the HTTP connections are typcially short-lived and can most likely be gracefully terminated to induce a new connection when retirement of CIDs is necessary), my preference goes to (c): an endpoint that uses RTP should cover the cost.
NAME (I think you mean RPT, not RTP) Thank you for the analysis. I prefer (c) as well, except that I do not want to condition it on non-zero RPT, to keep things simpler. Otherwise we're still left with what to do at the receiver. Ensuring that the handshake confirmation is acked means that there are no HS packets pending.
I'll also note that this issue was found through a resilience test we've built for the interop runner -- high loss handshake test. We're excited to see how everyone's handshake machinery performs under 50% loss :-)
At the time of this writing, ngtcp2 does (b).
NAME for you problem with (a) isn't that just a general problem with having multiple paths and then getting a NEWCONNECTIONID with RPT for all previous CIDs? You end up in a scenario where you only have 1 CID (the new one) and multiple paths that just had all their CIDs retired. If you can get away with using that one CID for the active path, you can still make forward progress. If not, the connection dies.
I believe we specify (b) already: "A client MUST only change the value it sends in the Destination Connection ID in response to the first packet of each type it receives from the server (Retry or Initial); a server MUST set its value based on the Initial packet. Any additional changes are not permitted; if subsequent packets of those types include a different Source Connection ID, they MUST be discarded. This avoids problems that might arise from stateless processing of multiple Initial packets producing different connection IDs." URL
NAME that is just indicating when a client should change DCID in response to receiving a Initial/Retry packet with a new CID. I don't think anything restricts either side from sending NEWCONNECTIONID frames in their first 1-RTT packets and their peers immediately using them for any packets (including handshake) they send. Personally, I don't have a problem with (a). I think that no matter what receivers will have to handle the multiple paths but single CID scenario, so I would prefer not to special case the handshake. I also don't think we have handshake confirmation problems because an endpoint should only send a NEWCONNECTIONID out when it's able to accept it. Then, if the peer can decrypt the packet with the new CID, I see no reason why it shouldn't be allowed to use it immediately.
NAME my reading of the text I quoted is that you cannot use any DCIDs in long header packets that weren't either the original client DCID/SCID or from a server Initial or Retry.
NAME There's a fourth option: d) require that the sender withholds the NCID frame until it has confirmed the handshake. As soon as an endpoint confirms the handshake, it drops the Handshake keys (I'm assuming here). As a result, the endpoint obviously doesn't care about Handshake packets any longer (and especially doesn't care which DCID was used to send the packet). Therefore, there's no problem if the receiver of the NCID frame immediately retires the connection ID used during the handshake, even if the the NCID and the HANDSHAKE_DONE frame were reordered.
NAME The part about what to do with DCID changes is underspecified. From the part of the spec you quote -- "Any additional changes are not permitted; if subsequent packets of those types include a different Source Connection ID, they MUST be discarded." -- it's not clear what these additional changes are limited to. I think the intent was for all long-form packets, but that's certainly not clear here. In any case, the problem with that approach is that the client will have to carefully construct HS packets with the old DCID while sending everything else with the new DCID.
NAME : (d) works too, but an endpoint still has to decide which DCID to use for sending HS packets in the case that the HANDSHAKE_DONE frame is lost. I'd rather not leave that unspecified.
"of those types" => Retry or Initial, since that's what it was talking about. That's trying to avoid the case where the server statelessly generates a new Retry (or Initial, in the rare case that would be stateless) with a second SCID upon receiving a retransmission or duplication of the client's Initial. But it doesn't say anything about the server issuing new CIDs via an NCID frame in 1-RTT before the handshake is confirmed. I think (a) or (b) would be fine; the server is clearly intending to accept the other CID if it issued it, but could conceivably use separate logic for long-header versus short-header routing.
Can someone explain why using the new connection ID is a problem?
NAME I think that that is a good way at looking into the issue. I agree that the endpoints can start using the new CIDs during handshake, if the peer sends NCID frames. The only problem that I can think of is related to retirement (see URL). When the server provides TP.preferred_address, and also decides to retire the CIDs immediately after sending all the handshake transcript, there's going to be a very small time window in which it has to send two CIDs (one for the current path and another for the path using the preferred address). Unless the QUIC stack allows bundling of two NCID frames when building a QUIC packet, the client might end up not having sufficient amount of CIDs, in which case it might terminate the connection. It is my view that this is very different from the ordinary case of retiring CIDs, in which case there would be enough time window to guarantee that a handful number of new generation CIDs are delivered before the older ones are delivered (cc NAME That said, I now tend to think that the most we might have to do is the following two: clarify that CIDs received using NCID frames can be used for Handshake packets have a cautionary text stating that a server needs to make sure that the client has enough number of new generation CIDs when it retires an older generation of CIDs
NAME Wouldn't we be able to solve this by using my proposed option d?
Artificial constraints on when the frame can be sent would be terrible to enforce and would introduce a performance cost for using NEWCONNECTIONID. In a great many cases, the 0.5RTT from the server is idle, which makes it a great way to get things like NEWCONNECTIONID sent without taking capacity from real work. Asking the server to defer sending would mean that NEWCONNECTIONID could compete with HTTP responses and the like. I see no reason not to allow use of connection IDs when they are available. They apply to the connection as a whole. You will note that we allow changing them with the Initial/Retry mini-protocol in ways that don't correlate with other connection-level events, so this isn't any different in that regard. Forcing the connection ID state to synchronize with other state changes seems more likely to complicate things than help. The caution NAME mentions is always true. Whenever an endpoint is forced to retire connection IDs, a short supply is always a risk to the connection.
I think QUIC allows more than 1 NCID frames in a single QUIC packet. Now my preference is (a). If server utilizes retire-prior-to, it should know the consequences and should provide sufficient backup connection IDs. Otherwise, it would be misconfiguration of server software.
NAME The problem is that the client might end up receiving only some of the NCID frames, unless the server always sends (and retransmits) the set of NCID frames as a whole. But anyways, I agree that (a) is probably fine. That is because it is trivial to design a server that never requests CID retirement during the handshake. All you need to do is start issuing the new generation of CIDs at least N seconds before sending NCID frames that asks for the retirement of the previous generation, where N is your handshake timeout. Knowing that this design is why I initially preferred (c), not realizing that it works equally well with (a). But know that I understand that, I prefer (a) as it is simpler.
Given a server has to sent NCID to have the CID change, I'm fine with (a), but I thought it was not what we'd previously agreed to and specified, so I believed (b) was the status quo.
I don't think this is a serious problem, because if the server doesn't want the long header DCID to change, it can wait before sending NEWCONNECTIONID. That said, we do have an issue here in that different readers of the draft had different interpretations of what was allowed. All that to say I don't much care if this is allowed or prohibited, but we should at least make an editorial change to clarify things.
I agree with NAME here. Any alternative is ok, but the draft should say whether servers can expect that the client might switch DCID and whether the clients can expect the server to send NCID w/ RPT>0 before the handshake is complete.
Let's keep it simple: no migration before Handshake Done, period. That would include prohibit NCID in 0-RTT Also state that implementations MAY drop any packet received with NCID different from original before Handshake Done.
NAME The problem is not that simple. Consider the case where the server receives ClientFinished and sends two 1-RTT packets, the first packet containing HANDSHAKE_DONE, and the second packet containing an NCID with RPT set to 1. If the first 1-RTT packet gets lost, the client receives a request to retire the connection ID before it confirms the handshake.
I think I'd prefer a design that meets the two requirements stated below: A server should not be required to withhold NCID frames until it receives an ACK for HANDSHAKEDONE. A client should be allowed to use the original CID until the handshake is confirmed. The intention is to minimize additional complexity to existing (and future) QUIC stacks. One solution that meets these requirements is to state that a NCID frame confirms the handshake, exactly the same way as HANDSHAKEDONE frame does. That would be a trivial change to any of the QUIC stacks, and there would be no ambiguity regarding the state changes.
Discussed in ZRH. Proposed resolution is to clarify that issuing a CID means being prepared to seeing it used at any time. Additional Editorial issue on packetization issues.
Editorial PR on packetization challenges with retransmitted frames is .
Ugh, editorial changes for a design issue. Reopened for consensus calling.
This was fixed by . is related to this issue, but that simply clarifies things. To keep things sane, let's just close this when is merged.
Looks good. URL discovered a minor "discrepancy" in our requirements for coalescing packets. We required that: senders not coalesce packets for different connections in the same datagram. receivers reject packets with different connection IDs in the same datagram. URL was the result of a discussion where I was convinced that looser constraints here weren't harmful. The requirements would identify connections, not connection IDs. Originally, my inclination was to fix the requirement on the sender to use connection IDs, which would remove some linkability. But then I was reminded that linkability is already defeated for several reasons. You can't coalesce outside of the handshake and we require a stable path for the handshake AND long headers include a source connection ID that can be used for linkability. So the linkability concern is basically lost. We also learned that some implementations were generating packets while processing incoming packets. Later they would coalesce these into a datagram that might then have different connection IDs on the packets it contains. There has been some opposition to the proposed resolution in PR 3870. Apparently, for some, having multiple connection IDs in the same datagram complicates processing. I don't understand this objection. It seems to me more difficult to retain state across packets than it is to process each atomically. I was hoping that Christian or Nick can explain more about how this affects them. Marten also indicated that there was a reason not to allow this, but if I understand that, this is based on some potential for future optimization, which could be conditional on having consistent connection IDs. If we moved to a scheme that required consistent connection IDs, anyone using that scheme could be required to avoid coalescing different connection IDs. So this seems more of a theoretical concern. And - in anticipation of maybe choosing to make senders use a consistent connection ID - if you might be generating coalesced datagrams with different connection IDs in packets, how hard would it be to ensure that these are consistent? Lars and Ian, I think both of you indicated that you might do this.

carry this connection ID for the duration of the connection or until its peer invalidates the connection ID via a RETIRE_CONNECTION_ID frame (frame-retire-connection-id).  Connection IDs that are issued  <ins> and not retired are considered active; any active connection ID is valid for use at any time, in any packet type.  This includes the connection ID issued by the server via the preferred_address transport parameter. </ins> An endpoint SHOULD ensure that its peer has a sufficient number of available and unused connection IDs.  Endpoints store received


All HTTP/3 requests MUST include exactly one value for the ":method", ":scheme", and ":path" pseudo-header fields, unless it is a CONNECT <del> request (connect).  An HTTP request that omits mandatory pseudo- header fields or contains invalid values for those fields is malformed (malformed). </del>  HTTP/3 does not define a way to carry the version identifier that is included in the HTTP/1.1 request line.

From an interop discussion with NAME and NAME HTTP/1.1 the presence of a Host header: If no Host header is present, the section on describes how you determine what host is being asked for, ending with "guess" if it's an HTTP/1.0 request. HTTP/2 says that you generate an pseudo-header if sending direct-to-H2 and carry through the Host header if you're relaying HTTP/1.1. There is no discussion about what to do with an HTTP/1.0 (or 0.9) request being relayed over HTTP/2, which wouldn't have a Host header; you're neither required to generate or send one, even an empty one, nor are you given guidance about how to figure it out. HTTP/3 carries forward the H2 language (currently by reference, and explicitly after ). Do we want to have some guidance here? Or does this issue instead belong in URL to move some of the existing guidance to be HTTP version-independent? (NAME
NAME Thank you for looking into the problem. Regarding if (or how) we should handle the issue, I think my weak preference goes to having one rule that applies to both HTTP/2 and HTTP/3. That would be less surprising to the users, and would also be helpful to implementations. It could well be the case for an HTTP server to have a common logic of handing pseudo headers (or lack of) between HTTP/2 and HTTP/3.
Host is an HTTP/1.x header field (it was just late to the party), so HTTP/1.0 includes it as an extension. And that requirement is specific to HTTP/1.1, so would theoretically not apply to HTTP/1.2 (and that was intended): it was a short term requirement imposed by the IESG because they wanted to require deployment and didn't understand that it was already deployed. In general, I don't understand why ":authority" exists (as opposed to always using ":host"), but I think the HTTP requirement is simply that the identifying data must be present somewhere in the request.
":authority" exists for the purpose of always transporting the equivalent of the , with ":scheme" and ":path" transporting the rest. It's mostly a consistency thing, I think. If you have a "Host" header instead, that's acceptable, too. So should we say that the target authority MUST be present in either "Host" or ":authority", or do we leave it alone?
Well, assuming we still allow requests for URIs that do not have an authority component (e.g, a URN or similar), I think it is better to just leave it alone. OTOH, folks are still going to ask what needs to be sent when the request target doesn't have an authority component, so that ought to be defined somewhere here more sensibly than it was for 1.1.
Discussed in ZRH. Proposed resolution is that if a URI scheme has a mandatory component, it better have one of those two things, preferably a :authority. Alert httpbis.
Doesn't this beg the question "what if the request is for a URI that does not have a mandatory authority component?" All of the other cases are described. I would add something like: Thanks!

All HTTP/3 requests MUST include exactly one value for the ":method", ":scheme", and ":path" pseudo-header fields, unless it is a CONNECT  <ins> request (connect). If the ":scheme" pseudo-header field identifies a scheme which has a mandatory authority component (including "http" and "https"), the request MUST contain either an ":authority" pseudo-header field or a "Host" header field.  If these fields are present, they MUST NOT be empty.  If both fields are present, they MUST contain the same value. If the scheme does not have a mandatory authority component and none is provided in the request target, the request MUST NOT contain the ":authority" pseudo-header and "Host" header fields. An HTTP request that omits mandatory pseudo-header fields or contains invalid values for those fields is malformed (malformed). </ins> HTTP/3 does not define a way to carry the version identifier that is included in the HTTP/1.1 request line.


A packet that does not increase the largest received packet number for its packet number space (packet-numbers) by exactly one.  A <del> packet can arrive out of order if it is delayed or if earlier packets are lost or delayed. </del>  An entity that can participate in a QUIC connection by generating, receiving, and processing QUIC packets.  There are only two types

A sender can intentionally skip a packet number, causing the receiver to treat a packet as "Out-of-order". From NAME comment on

A packet that does not increase the largest received packet number for its packet number space (packet-numbers) by exactly one.  A  <ins> packet can arrive out of order if it is delayed, if earlier packets are lost or delayed, or if the sender intentionally skips a packet number. </ins> An entity that can participate in a QUIC connection by generating, receiving, and processing QUIC packets.  There are only two types


Servers MUST drop incoming packets under all other circumstances.  5.3. A QUIC connection is a stateful interaction between a client and

Addresses . I'm flexible on wording and where exactly it lives in the document. I put it in Security Considerations.
I just completely rearranged the new section based on your input. See how it reads now. Editors, I don't seem to have permission to request review. Can someone add NAME
Unfortunately, some of the reviews are hidden as obsolete. I want to respond to this by NAME about disablemigration I tried to not recommend anything here except to send disablemigration. The client is going to decide what to do with regard to NAT rebinding. If wired, I probably send more PINGs or something. If wireless, maybe not. If I'm quite sure I'm not behind a NAT, I probably do nothing. There's nothing normative here but to send the TP, so if people hate this I can just delete the explanation.
I moved the whole thing to Section 5 as NAME suggested. I've taken most of the other suggestions, but am not sure if the intro needs to change, or what exactly people don't like about the disable_migration explanatory text.
I'm actually fine with this. We were considering keep alives on the server side when we can't support migration (or NAT rebinding) based on the current infrastructure. So having the spec suggest keep alives to the client would be better. It would allow the client to choose if it would prefer keep alives or just to close the connection.
After taking all suggestions from NAME I simply deleted the explanatory text about sending disablemigration. I didn't see the stuff about NAT rebinding as prescriptive or changing the semantics, but it's non-normative, clients are gonna do what they gonna do, and it's not worth holding up the rest of this over. If people want to add NAT rebinding considerations to disablemigration we can fight that out in another issue.
FWIW, we have a number of cases when the destination IP and port + CID would route correctly, but the original IP(being anycast) would not always route correctly. It's likely there are millions of connections to the destination IP being migrated to, but yes it's a smaller number than the anycast IP.
So NAME do you want new or revised language here, or will the other issue address the linkability stuff? I would like to close this out if it's good enough.
Given how hard this has been to get right, I want NAME to take a look. But I'm happy with this.
A reminder for NAME to either take a look or say that this is OK to proceed without another review.
I could not stand leaving this unmerged any longer. Anyone with more comments can make a new PR. Thanks to everyone who helped out here.
The transport draft talks about QUIC being resilient to NAT rebindings. However, if a QUIC server is behind an L3 load balancer which simply routes based on 5-tuple, then connections to this server will (likely) not survive rebinding. I couldn't find any language which addressed whether such a deployment was "OK" or not. I think that since this load balance does not support NAT binding resilience, it is implicitly "bad" according to the draft, but others might disagree. In any case, I think there should be text to address this. Note, if the server advertised a preferredaddress which routed around the load balancer, and if clients were required to use this address, then that could obviously work. But preferredaddress support is a SHOULD, not a MUST.
We did discuss this issue. Use of empty connection IDs and demultiplexing based on source addresses. The conclusion there was that this was OK: people could do that, as long as it was clear that what they were getting was not different than TCP. That is, we made it clear that if you don't use the identifiers that you control (your addresses, the connection ID), then you accept that you can't handle migration at all and connections will drop (well, unless you do something like trial decryption, which has some obvious scaling issues). The outcome of that discussion was captured in . Assuming that Ryan finds this answer satisfactory (and I haven't missed anything), then I suggest we just close this as a duplicate of .
I don't think this captures Ryan's concern: when the server is using a non-zero CID and also is behind 4-tuple routing. I think he's looking for a section of text to coherently describe what you have to do here: send either disablemigration or preferredaddress; or forward packets between servers; or either don't use a common stateless_reset key, or put client address/port in the reset token. This is sort of ops-drafty but there real requirements on servers to behave in a secure way. I think someone with full command of the transport draft would figure this out, but it is not clearly stated anywhere.
I agree with Martin Duke and the language he suggests makes sense to me.
This does feel a bit ops-drafty to me, but I'd be happy to review a PR.
I'm sure that this is now basically unrecognizable...Some more suggestions, but overall this LG

Servers MUST drop incoming packets under all other circumstances. <ins> 5.2.3. A server deployment could load balance among servers using only source and destination IP addresses and ports.  Changes to the client's IP address or port could result in packets being forwarded to the wrong server.  Such a server deployment could use one of the following methods for connection continuity when a client's address changes. Servers could use an out-of-band mechanism to forward packets to the correct server based on Connection ID. If servers can use other dedicated server IP addresses or ports than the one that the client initially connects to, they could use the preferred_address transport parameter to request that clients move connections to these dedicated addresses.  Note that clients could choose not to use the preferred address. A server in a deployment that does not implement a solution to maintain connection continuity during connection migration SHOULD disallow migration using the disable_active_migration transport parameter. Server deployments that use this simple form of load balancing MUST avoid the creation of a stateless reset oracle; see reset-oracle. </ins> 5.3. A QUIC connection is a stateful interaction between a client and


in connection failures, as the issuing endpoint might be unable to continue using the connection IDs with the active connection.  5.2. Incoming packets are classified on receipt.  Packets can either be

This is attempt to write down NAME 's proposal to This was surprisingly hard to write, and undoubtedly could be improved. There are cleaner ways to solve this problem, but this one is lowest-impact to the existing spec.
NAME thanks for the new text, that helped. I think we've resolved outstanding issues, unless my new text has created new problems. 2 possible ways to enlarge this solution: should we describe this attack in Security Considerations? As these MAYs are an effort to counter something in particular, perhaps we should discuss. it would clean up some accounting issues if each NCID frame MUST set Retire Prior To to more than the maximum continuous sequence number that it had recieved an RCID for. Worthwhile?
That's interesting to send at first glance, but then poses the question of what the recipient of NCID should do when it sees an RPT value change. There's a difference in behavior between "I've seen these" and "you need to send these."
I think this PR is sufficient to both document the issue and provide some minimum guidance, so I'd suggest we editorialize a bit and then move forward with a solution along these lines.
It seems all the discussion of the problem is happening here and not on the issue, so I will propose my alternative solution here: I know folks have been resistant to changing state based off receiving an acknowledgement of a packet/frame they sent, but what if we changed how the RetirePriorTo works, and just required the packet containing the frame to be acknowledged, instead of requiring the RCID to be sent? Would this solve the problems? This would mean the receiver of the NCID shouldn't have to track any additional state beyond what it takes to acknowledge the packet (which they have to do anyways); it just immediately throws away the state. And the RCID packet would only be used when the endpoint that was given the CID chooses to retire it.
NAME I think this PR has been an attempt to clarify the "leaky" approach, that tries to clarify the amount of state that an endpoint should retain. There are indeed discussion of what that amount is, but I think it is essential to have such detailed discussion on the PR in order to determine exactly what a particular approach would be like. To paraphrase, I think we are making process on this PR to describe a specific approach. If we are to consider something different, I think that discussion should happen on the issue (and emerge as a separate PR).
Ok, trying to put this one in the ground. I think the only remaining dispute is between NAME and NAME on whether activeconnectionid_limit is a sufficient number of RCIDs to remember.
Reading through this proposal, I fear that this would be very difficult to implement. If the ACK for a RETIRECONNECTIONID frame is lost, and endpoint cannot safely send a NEWCONNECTIONID frame with a RetirePriorTo, since this would cause the peer to have more than RETIRECONNECTION_ID frames in flight. In order to safely use RetirePriorTo, this endpoint then would have to track receipt of the ACK frame. I'd very much like to avoid the complexity associated with that.
I finally caught up with this issue, and I think that implementer advice is my preferred way forward. To that end, I like this PR as the resolution.
OK -- there has been an enormous amount of discussion. The current state of the PR is that all RCIDs will eventually be sent, but if there are more than activeconnectionid_limit, some may be delayed by an RTT or more. If it gets out of hand the RCID sender can throw a connection error. I believe this reflects what NAME articulated a few dozen messages ago, and in the thread above I have pseudocode for a lightweight implementation of this (i.e. essentially no additional resources for an arbitrarily large backlog of RCIDs). The advantage of this approach is that it is lightweight and is very subtle change to behavior that will not will not affect interoperability at all or break the existing contract of a Retire Prior To. There are two alternatives that have some traction: 1) Just bite the bullet and do . This breaks interoperability but is a clean architectural change that addresses the root of the problem. I would be very comfortable with this, and prefer it to other things that break the existing semantics. 2) NAME "leaky" approach that allows endpoints to simply not send some RCIDs at all if there are too many pending. I used to prefer this, but given my existence proof above I'm not as excited about it anymore. It breaks the contract that is in draft-27 instead of merely bending it. I hope that covers the decision space. I would like to put this issue out of its misery.
I'm reluctant to blow away the whole PR with brand new text, so in this comment I will propose four entirely new paragraphs that IMO says what needs to be said. Either the current PR, Martin's revision, or this proposal could serve as the basis of the eventual change: If a peer sends large numbers of NEWCONNECTIONID frames that increase Retire Prior To, and/or acks of packets that contain RETIRECONNECTIONID are lost, the state required at the RETIRECONNECTIONID sender can grow without regard to its activeconnectionidlimit. Therefore, endpoints SHOULD take steps to bound the state associated with needed RETIRECONNECTIONID frames while ensuring that it eventually transmits all required RETIRECONNECTIONID frames. For example, it might limit voluntarily retirement of sequence numbers if it has not received enough acknowledgments of packets containing previous retirements. It might also restrict the RETIRECONNECTIONID frames in flight to a single packet in order to simplify tracking of what is in flight, what needs retransmission, and what has been acknowledged. An endpoint MAY treat having too many connection IDs to retire as a connection error of type CONNECTIONIDLIMITERROR. The threshold for this error SHOULD be at least twice the endpoint's advertised activeconnectionidlimit. Endpoints SHOULD NOT issue updates of the Retire Prior To field prior to receiving all of the RETIRECONNECTION_ID frames for the previous update to Retire Prior To. ** If people much prefer this or MT's revision, I'll update the PR accordingly.
I took NAME 's suggested text. I would prefer a little advice on how to implement the lag, but I can live with this text.
LGTM.
When an endpoint sends a RETIRECONNECTIONID frame carrying a connection ID that has been active until that point, that endpoint is expected to open room for accepting a new connection ID. That means that if the peer does not acknowledge the packets that contains RETIRECONNECTIONID frames, but keeps on sending NEWCONNECTIONID frames with increasing values in the Retire Prior To field, the number of unacknowledged RETIRECONNECTIONID frames (or to be precise, the number of unacked retiring connection IDs) keeps on increasing.
To address the issue, I think we need to do one of the following: State that the attack exists, and that QUIC stacks should implement mitigations. Introduce MAXCONNECTIONIDS frame. The reason we do not have similar issue in stream concurrency control is because the credit is controlled by MAX_STREAMS frames, rather than using FIN (or reset) as an implicit signal to indicate availability of new credit. We could do the same for connection IDs.
Mathematically, this might be unbounded, but it's more likely that in practice the limit is as it would be unlikely that unacknowledged RETIRECONNECTIONID frames would remain so once more NEWCONNECTIONID frames start arriving. More so as the issuer of those NEWCONNECTIONID frames has to have received the RETIRECONNECTION_ID frames. Adding new frames for this seems a little unwieldy, so I'd prefer to just acknowledge that the problem exists. Note also that Retire Prior To has the effect of making this limit squishy in other ways. (This is a design issue.)
I'm not sure I understand the attack here. Why would an implementation need to keep track of acknowledgements of RETIRECONNECTIONID frames (other than for purposes of loss recovery, but then this applies to any retransmittable frame type)? An endpoint sends a RETIRECONNECTIONID frame when it decides to not use the respective CID any longer. At this point, it can forget about the connection ID altogether. It might make sense to hold on to the stateless reset token for a while (3 PTO?) longer, in order to be able to detect stateless resets, but this is an optimization and wouldn't lead to unbounded state anyway.
NAME I think is fine, assuming that the endpoint would just stop retransmitting oldest RETIRECONNECTIONID frames. That limit would be too stern to be used for detecting potential attacks (and therefore closing the connection with PROTOCOLVIOLATION).
NAME This is about keeping state for retransmission for recovering loss. As stated in , the issue is unique to RETIRECONNECTIONID frames, because this frame type is (ab)used for providing the peer additional credit, compared to other flow control that uses dedicated frame types (e.g., MAX_*) for communicating credit.
Thank you for the explanation, NAME I think I now understood the problem. This would require endpoints to make assumptions about how the peer handles the and how it issues new CIDs. This can get complicated really quickly and leave the connection in an unrecoverable state (or at the least a state with less CIDs than it should have). Introducing a new frame, as NAME suggested, seems a lot cleaner to me.
Just to be clear, while I used RPT as a way of explaining the design issue, I think it is not a requirement. A malicious client can repeatedly migrate to a new address, initiating the use of new CID pairs, at the same time intentionally not acknowledging packets containing RETIRECONNECTIONID frames sent from the server. If the client does that, the number of CIDs that the server has to track for retirement increases as time goes.
Clarification question: If the person sending RCID isn't getting ACKs back, I think we still need to assume that those CIDs are being replaced? If not, then the non-RPT side of this would just mean the person sending RCID frames runs out of CIDs to use and has to stop. So the problem statement is that: this person is running over their maxactivecid limit from the perspective of the person withholding ACKs, but each side has their own view and so the person retiring them stopped counting them as active once the frame was sent (but not acknowledged). And that can happen via RPT as well, just because it means that they're going to be instructed to send the RCID, but the problem is the same either way? (Just want to make sure I'm understanding the problem correctly)
NAME Yes, I think your problem statement is correct. Or to be even more concise, the problem is that the issuer of CIDs is given additional credit when it receives RCID, which happens before the consumer of CIDs drops the state it needs to retain for retransmitting RCID. For the attack to work, an attacker has to let the peer consume and retire CIDs. RPT and intentional migration are examples that makes that happen.
So the issue here is fundamentally that you can induce the peer to send retransmittable frames which aren't subject to some kind of flow control, and this is the only instance we have of that. STOPSENDING and failing to acknowledge the RESETSTREAM appears to have a similar profile, except that the number of RESETSTREAMs you could have outstanding is bounded by the number of streams the peer has opened, which they control. You could execute the same attack at a larger scale by choosing particular regions of stream data to never acknowledge, forcing the peer to keep them in memory for retransmission, but the peer could stop sending until that region gets through (applying backpressure) or simply reset the stream. Since a RESETSTREAM is smaller than the data region and only one is needed per stream, this collapses into the previous case. What's different here is that the endpoint being attacked doesn't choose whether to retire these CIDs and has no mechanism to apply backpressure. It seems like there are a couple paths to resolving this. As you suggest, we could define an explicit frame that grants the additional CID space, and implementations would not send / could withhold this frame until the RCID is ACK'd. This creates a backpressure mechanism and has the side benefit of allowing an implementation to vary the number of CIDs it wants to track over time, but the disadvantage of requiring an extra round trip before CIDs can be replaced in the normal case. We could mimic the "collapsing" behavior above by defining a frame that retires all CIDs up to a given sequence number. Then the implementation under memory pressure chooses to retire a few extra CIDs and collapses the whole set waiting to be retransmitted to a single frame.
NAME I think your summarization of the problem is accurate. Regarding an attacker selectively acking packets carrying STREAM frames, yes, the attack exists, but it the sender can mitigate that problem in certain ways. We already state that in . This issue is the only case that a protocol cannot be implemented properly, with a bounded amount of state. Regarding how we should fix the problem, if we are going make a protocol change, my preference goes to applying the same design principle that we have in other places (i.e. introduce MAXCONNECTIONSIDS frame). I do not like designing a mechanism specific for this purpose, as there is a chance that we might make mistakes. I can also live with what NAME proposed in URL (i.e. encourage leaky behavior).
Similar to NAME I'd prefer to acknowledge the problem and indicate that peers may want to limit the number of unacknowledged RETIRECONNECTIONID frames to the number of connection IDs they support. The other option I can imagine is to make the RETIRECONNECTIONID frame cumulative, which provides a strong incentive to use connection IDs in order. Though we may have to make an exception for the connection ID in the received packet if we did that.
So the bounds on this aren't that terrible. Say you had to retire 1000 connection IDs all at once. You could track each one individually as you send RETIRECONNECTIONID. That's 1000 things to track, which might be more state than you want (it's not 1000 bits because you need to maintain relationships with packet numbers). Or you could just keep strictly limited, no matter how many connection IDs you need to retire. Tracking that, plus what you have been requested to retire should bound the state required to implement retirement. In other words, you do control how much state you commit to retirement, just like everything else.
NAME That's my preferred solution, but in my reading of the spec this is not allowed. We MUST send RCID for each sequence number for which the peer has sent retire-prior-to. I think the language in Section 5.1.2 needs to change.
NAME idea of allowing RCID to be cumulative is also good and computationally cleaner, although we'd need a different frame type or an additional field in the frame (I don't think we can get rid of retiring non-continuous sequence numbers)
I'm happy to file a PR for the NAME or NAME approach based on the overall sentiment.
One purely implementation option: It's always legitimate to repeat a frame. Therefore, if you track only the smallest and largest un-acked RCID sequence numbers, you can just keep retiring the whole range every time you retransmit. ACKs will shrink the range (hopefully to zero). There's no harm beyond unnecessary bytes in retransmitting something a few extra times to avoid excessive state tracking.
NAME While I won't say that's impossible, I'd argue that the design is going to be complex. When CIDs are used in a legitimate manner, they are not going to be retired one by one. A CID is retired when a path using that CID is abandoned. Consider the case where an endpoint has 4 active CIDs (CID0 to CID3), and two paths. One path could be using CID0, the other path could be using CID3. When the endpoint abandon the path that is using CID3, it has to retire CID3 but it cannot retire others. NAME The amount of CIDs that can be sent on wire at a time is restricted by the CWND, therefore there is no guarantee that you can send RCID frames for all CIDs that you want to retire at once. In other words, I think the approach at best collapses into what NAME has proposed (i.e. leaky behavior).
At this point, I also think NAME solution is the best approach, since it is a very minor change, and changing RETIRECONNECTIONID to be cumulative is a larger change than I want to accept at this point. NAME thanks for your examples.
One idea: Can we simplify everything by not requiring sending RETIRECONNECTIONID for CIDs less than the Retire Prior To? RETIRECONNECTIONID would then only be used when a CID was used and needed to be retired so more CIDs could be issued to the user of the CID.
NAME see my comment in the PR. More disruptive to current implementations, and eliminates a useful signal. The current proposal will not matter for well-behaved clients but will protect servers from some attacks.
We haven't implemented this yet, so it's not disruptive to our implementation FWIW I wonder how useful the signal is. Retire Prior To is really saying "These are going away sometime soon, whether you like it or not." to the peer. If an implementation never sent RETIRECONNECTIONID, would that create a problem of some sort, because I suspect someone will do just that.
NAME As stated in URL, this attack can be mounted without using Retire Prior To.
NAME in this case isn't the client limited by activeconnectionid_limit? As a server, if the client has in no way indicated it has accepted my retirement, I'm not going to accept new CIDs. The NCID/RPT case is different because the spec specifically allows the client to NCID speculatively assuming the server will immediately retire the CID.
NAME The attack here is that when a malicious client retires a CID to which the server has responded, that server would retire the CID that it has used on that path, and also provide a new CID to the client. But the client never ACKs the RCID frames that the server sends. As an example, consider the case where a client uses sCID1 on a new path, the server responds on that path using cCID1, the client abandons that path and sends RCID (seq=sCID1). When receiving this RCID frame, the server would send RCID (seq=cCID1), and also send NCID(sCID2). The client intentionally does not ack the packet carrying these frames, but uses sCID2 on a new path, that carries NCID(cCID2), repeating this procedure.
Ah, I see now that my defense doesn't work, because if I reject the NCID I have to kill the connection, and it may just be because of unlucky ack losses. This whole section is not very precisely written and suffers from two-generals problems about when a CID is actually retired. I will revise to account for this, but I'm not sure how.
NAME I agree with NAME that there's no valid thread model without Retire Prior To. If the peer doesn't acknowledge your retirement, then you shouldn't be issuing more CIDs, because you might exceed their limit.
In my opinion, running out of CIDs is not a big crisis. You can always create a new connection with 0-RTT with a relatively small overhead. We should be biased towards simplicity rather than never running out of CIDs.
NAME unfortunately we are cornered by the spec. Consider two well behaving endpoints. I'm fully stocked with CIDs and then retire one with a frame. The peer gets the RCID and issues an NCID, but the ack of RCID is lost. If I don't count the CID as retired, then I MUST close the connection. That's bad, so I have to treat the CID as retired if I sent RCID. If you drop acks at scale, this becomes an attack. I'm really coming around to your cumulative retire idea, but will have another go at this tomorrow.
NAME I think the framing is incorrect. The peer can enforce an endpoint to retire a CID (e.g., by changing RPT or discarding a path). As NAME points out, an endpoint is encouraged issue a new CID at this point, rather than withholding than until the ACK of RCID is received. Note also that RCID that an endpoint sends carries the CID that the peer has issued, while NCID that the endpoint carries the CID that that endpoint issues. Therefore, I am not sure what the rationale for withholding the emission of NCID until receiving an ACK for RCID would be. I agree. An endpoint is not expected to consume CID at high rate. We already discourage such behavior; see URL That's why I'm fine with the leaky behavior that NAME has suggested. NAME As previously stated, I am hesitant to have a new machinery that is unique to this purpose. There is a risk of creating something buggy, both in terms of specification and implementation. My preference goes to accepting the risk (and recommending mitigations, such as the leaky behavior), or reusing a design pattern that we already have (i.e. introduce MAXCONNECTIONIDS frame).
The cumulative idea is very appealing to me, minus a future with multipath. If two or more paths are active at once, I don't know how the peer would know that if the frame was cumulative. The only option would be to ensure the two(or more) CIDs in use are adjacent, which is possible, but may require switching CIDs more than should be necessary.
NAME I would suggest a cumulative RCID as a supplement to the individual RCID. A cumulative one would cleanly handle a lot of cases where there is an unboundedly large list of sequence numbers to retire. If there is a low sequence number that is still in use, the sender can choose to retire it to enable use of the cumulative frame.
I know folks have been resistant to changing state based off receiving an acknowledgement of a packet/frame they sent, but what if we changed how the RetirePriorTo works, and just required the packet containing the frame to be acknowledged, instead of requiring the RCID to be sent? Would this solve the problems? This would mean the receiver of the NCID shouldn't have to track any additional state beyond what it takes to acknowledge the packet (which they have to do anyways); it just immediately throws away the state. And the RCID packet would only be used when the endpoint that was given the CID chooses to retire it.
FWIW, I have filed that fixes this issue without introducing a leaky behavior like . IMO, the observation is that the endpoints can check that the issuer of CIDs is not intentionally introducing gaps in the sequence numbers they generate. We already prohibit senders from introducing such gaps, detecting them on the receiver side would be a good thing to do. And if endpoints start doing that, using a cumulative RCID frame is a sensible solution, as the the cumulative list of retired CIDs at any time can be represented using a SACK-like structure (i.e., max-retired + small list of active CIDs below that max.).
I know that you have characterized this as "leaky", but I'm not sure that I understand exactly what is being leaked. Nothing is lost in , except that the endpoint requesting retirement has to tolerate a potentially extended period without old connection IDs being retired. In case this helps, I have no big preference between and . There is a certain elegance to that might tip me toward that, but it is slightly more disruptive. I prefer either of those to the more disruptive change in , which also has the downside of strongly encouraging connection ID changes in order to preserve a clean separation between active and retired. doesn't force you to drop old connection IDs. However, the encoding of the frame is just too complicated for me to accept. I would prefer forcing a clean separation than having a complex encoding like this. is a non-starter for me for the reasons stated.
NAME In the approach we take in , there is a possibility that the issuer of the CID might not receive the sequence numbers of all the CIDs that have been retired by the peer, when the issuer is issuing new CIDs at high rate and the receiver is also consuming them at a hight rate. That is the consequence of endpoints limiting the amount of inflight RCID frames it tracks. I think we can provide advice so that the "leak" would not be an issue in practice, and assuming that would happen, I am happy with being the solution. I think is cleaner, but I acknowledge that it is more disruptive than . Part of my intent behind creating is to see how a clean fix (in terms of state machinery) would look like, so that it can be compared against . As stated in URL, by itself only partly fixes the problem. I am not against adopting along with though. is also a partial fix, I also think that it is a non-starter.
I think that I understand your point; thanks for your patience. I didn't really classify it as a leak though, so the choice of words threw me a little, it's more of a lag. Let's say that you have a pathological endpoint that sends an infinite series of NEWCONNECTIONID frames with Sequence Number N and Retire Prior To of (N-1). That's legal and will never violate any limits. But unless they maintain acknowledgments for RETIRECONNECTIONID frames at a similar rate (which, to be fair should be easy as for the same values), they could get far ahead of their peer. I don't think that it is strictly a leak, just a lag between the connection IDs being pushed and the connection IDs being successfully retired. Without or something else this could be problematic. And I now understand why is really just an orthogonal refinement, though it might make other defenses less likely to be required by making zero in cases where an endpoint isn't using discretionary retirement. The defense in is to make effectively constant, no matter how many frames need to be retired. My main objection is really that it is a maximal design where only a minimal one is warranted. It's probably worth pointing out that there is also the simplest defense: If you can't keep up with RETIRECONNECTIONID, stop talking to the peer that does this to you. This is just another example from that category of abuse we don't have specific protocol mechanisms to mitigate.
I agree that I'd like to see a minimal solution, so I'm leery of heading towards . In order for a peer to need to send a lot of RETIRECONNECTIONID frames without Retire Prior To, a lot of 5-tuple + CID changes need to be coming in. One way to rate limit that naturally is to stop giving out CIDs if the peer hasn't acknowledged your RETIRECONNECTIONID frames. ie: Limit the number of NEWCONNECTIONID frames in flight to: So if the peer stops acknowledging your retirements, you stop giving them new CIDs. I think this avoids the need for any other limits? I added 1 in my example so one NCID could be sent even if all of the peer's CIDs had just been retired, though that concern disappears if we decided to also do something like
What a mess we've made! Is there interest in doing a Zoom about this? Maybe 0900 Tokyo/1100 Sydney/1700 Seattle/2000 Boston on Tue/Wed? The root cause is that the flow control mechanism isn't all that well designed. IMO is the actual correct fix to this problem, but as people say, it's disruptive. uses ACKs as an implicit signal, which seems like a problem. has a different implicit signal, and I don't think it solves the case where the client is withholding acks for unsolicited RCIDs. I really think we should just use or depending on our appetite for changing the wire image. is great for the RCID sender, and if the receiver won't miss any RCIDs unless it's doing pathological stuff or the ack loss patterns are tremendously unlucky. , OTOH, is a comprehensive fix to the problem.
I think with some added text about limiting outstanding NCIDs when there are lots of RCIDs would be sufficiently robust.
I'm fine with trying to see if we can reach an agreement on .
I'd prefer over . I can't really see why it would more disruptive than . The problem with is that it will likely work find in most cases, but there are some corner case (when the ACKs for RETIRECONNECTIONID frames are lost), in which things will break. Properly accounting for these corner cases requires some non-trivial tracking logic in order to avoid risking the connection being closed with a CONNECTIONIDLIMIT_ERROR. On the other hand, seems to be conceptually simpler. I'd rather implement a few changes to my frame parser than introduce some ACK-tracking logic.
Just to give another example of how an endpoint might end up required to track the retirement of more than activeconnectionid_limit, when talking to a well-behaving peer: A server tries to supply three active CIDs to the peer. At the moment, it has provided CID0, CID1, CID2 to the client. The client sends RCID(CID1), RCID(CID2). The server provides NCID(CID3), NCID(CID4). The two NCID frames are delivered to the client, but the ACK for the RCID frames gets lost. The client decides to probe two paths concurrently, by using CID3 and CID4, but shortly after the packets are sent using those paths, the underlying network for those two paths disappear. The client now needs to make sure that RCID frames carrying 4 CIDs reach the peer. In this example, we might argue that it would be possible for the client to figure out that CID3 and CID4 were provided as substitutions for CID1 and CID2. But do we want to implement logic that detects such condition into our code? Note that tracking of substitutions can become tricky; if RCID(CID1) and RCID(CID2) were sent in different packets, it would be impossible for the client to see if CID4 was provided as a supplement for CID1 or for CID2.
I agree with NAME that if we go with the limit should be higher than .
I finally caught up with this issue, and implementer advice is definitely my preferred way forward. We're going to need some complexity to (and some magic numbers) no matter what, let's do this while limiting mechanism to a minimum. To that end, I agree with as a good resolution.
While increasing the limit will certainly help, this introduces a really unpleasant flakiness in the protocol. QUIC, so far, has the property that no amount or pattern of packet loss will lead to a protocol violation (it might lead to a connection timeout if all packets are blackholed, but that's a different kind of error). This means that when running QUIC in production, screening for (transport-level) errors will be very useful for finding bugs in implementations. My fear regarding is that no matter how high we choose the limit, there will always be a loss pattern that will trigger a protocol violation.
NAME Thank you for raising the concern, I think the recommended behavior when exceeding that limit is to stop sending RCID frames for some sequence numbers rather than triggering a protocol violation. That's actually what I had in mind hence called it "leaky." The benefit of stopping retiring some sequence numbers is that the worst case becomes endpoints running out of CIDs - something that is expected to happen on ordinary connections too. Note also that in HTTP/3, reestablishing a connection in 0-RTT is always a viable option. To summarize, I think that the concern can be resolved for HTTP/3 by changing the recommended behavior from resetting the connection to stop sending RCIF frames for some sequence numbers.
NAME I don't think not sending RCID frames is that easy. Tracking something that needs to be sent consumes comparable state to tracking it in flight. Or are you suggesting a different algorithm that indicates a RCID frame doesn't need to be sent? If the peer has to wait for one Retire Prior To to take effect before it's increased again, that leaves only the peer migrating really quickly and lost ACKs as the unbounded case. As NAME said, I think it's more likely a well-behaved peer would run out of CIDs in this case than any other failure mode. I previously suggested stopping sending out NCID frames when the RCID frame limit had been reached to force the issue, but that just changes who has to close the connection. FWIW, we already have sanity checks where our implementation closes the connection when a datastructure becomes too large. The limits are high enough that they're almost never hit unless there is a bug, but they've been helpful in finding bugs. I would advocate everyone have these. NAME You'll never get to 0 transport errors. Some implementations will have a bug, and some peers or servers will have faulty or corrupted memory. For example, in Google QUIC we found we received the ClientHello on a stream besides 1 surprisingly often.
I think that the idea is to reduce the set of RCID frames needed to a range of sequence numbers. You only have to remember the few exceptions for sequence numbers that are in use, sequence numbers for which you have outstanding frames, and a range of sequence numbers that require retirement. As long as your discretionary use is sequential, this is bounded state. As mentioned, the risk is that lack of acknowledgment leads to potentially large delays in getting new connection IDs, but if you ever run short, abandon the connection. Either you have a pathological loss pattern or your peer is messing with you. In either case, it is probably not worth continuing.
NAME Yes, this is one method of limiting the state that is being used for retaining sequence numbers that need to be sent. And it works even when discretionary use is not sequential, as the number of "exceptions" (i.e. gaps) is bounded by maxconnectionidlimit. The other assumption of using this method is that you would somehow bound the state that is used to track RCID frames being inflight. There are various ways of accomplishing that, e.g., have a limit on the number of RCID frames sent at once, track bunch of inflight RCID frames as one meta-frame. And it is correct that the outcome would be a "lag" in this method. The downside of this method is that it might be a disruptive change to existing implementations. It is essentially equivalent to what proposes, with the exception being that tracking RCID frames are difficult (as you track to send many). For existing implementations, it would be easier to adopt a "leaky" method, i.e.: let endpoints have a counter counting on the number of sequence numbers that are retired but are yet to be acknowledged when a CID is being retired, if the value of the counter is below a certain threshold (i.e. 2 maxconnectionidlimit), the counter is incremented, and a RCID frame carrying the sequence number of that CID is added to the outstanding queue. If the counter was not below the threshold, no RCID frame will be sent (leak!) when an RCID frame is deemed lost, it will be resubmitted into the outstanding queue when an RCID frame is acknowledged before deemed lost, the counter is decremented when an RCID frame is late-acked, we do nothing Assuming that the solution we prefer is the least-disruptive approach, I think we need to recommend the latter. If we have distaste in the latter and prefer the former, I think we'd better switch to , as it more cleanly reflects the design of the former. NAME I think the latter is not that a big change to existing implementations.
Yep, I think that is a good approach. Obviously, the range encoding is superior in terms of how much state it can hold, but a simple list is fine. And the effects of lag will be limited by the tolerance for retaining retired connection IDs, which seems proportional to me.
NAME I doubt that's less impactful on existing implementations, certainly from an interoperability standpoint. Today endpoints should hold on to incoming CIDs until they are formally retired. Making them leaky can lead to trapped state. Sending them all, but with a delay, is no added state but solves this problem.
After a lot of back-and-forth, I think that we've all generally converged on .
Still LG, but I like ekr's suggestionLatest text looks good

in connection failures, as the issuing endpoint might be unable to continue using the connection IDs with the active connection. <ins> An endpoint SHOULD limit the number of connection IDs it has retired locally and have not yet been acknowledged.  An endpoint SHOULD allow for sending and tracking a number of RETIRE_CONNECTION_ID frames of at least twice the active_connection_id limit.  An endpoint MUST NOT forget a connection ID without retiring it, though it MAY choose to treat having connection IDs in need of retirement that exceed this limit as a connection error of type CONNECTION_ID_LIMIT_ERROR. Endpoints SHOULD NOT issue updates of the Retire Prior To field before receiving RETIRE_CONNECTION_ID frames that retire all connection IDs indicated by the previous Retire Prior To value. </ins> 5.2. Incoming packets are classified on receipt.  Packets can either be


7. <del> TBD.  Also see Section 7.1 of RFC7541. </del>  While the negotiated limit on the dynamic table size accounts for much of the memory that can be consumed by a QPACK implementation,

Behold
NAME I think this is inches of getting landed
The full text of that section is "TBD."
HPACK covers the following topics: Probing Dynamic Table State Static Huffman Encoding Memory Consumption/DoS Implementation Limits To what degree to we need to retread this vs referring to 7541? There are some new areas to cover related to DoS, like recommending implementation timeouts for blocking requests. Soliciting any other topics, and possibly a passionate security-minded volunteer to draft a PR.
All looks good (and familiar); some minor nits.

7.  <ins> This section describes potential areas of security concern with QPACK: Use of compression as a length-based oracle for verifying guesses about secrets that are compressed into a shared compression context. Denial of service resulting from exhausting processing or memory capacity at a decoder. 7.1. QPACK reduces the length of header field encodings by exploiting the redundancy inherent in protocols like HTTP.  The ultimate goal of this is to reduce the amount of data that is required to send HTTP requests or responses. The compression context used to encode header fields can be probed by an attacker who can both define header fields to be encoded and transmitted and observe the length of those fields once they are encoded.  When an attacker can do both, they can adaptively modify requests in order to confirm guesses about the dynamic table state. If a guess is compressed into a shorter length, the attacker can observe the encoded length and infer that the guess was correct. This is possible even over the Transport Layer Security Protocol (TLS, see RFC5246), because while TLS provides confidentiality protection for content, it only provides a limited amount of protection for the length of that content. Padding schemes only provide limited protection against an attacker with these capabilities, potentially only forcing an increased number of guesses to learn the length associated with a given guess.  Padding schemes also work directly against compression by increasing the number of bits that are transmitted. Attacks like CRIME CRIME demonstrated the existence of these general attacker capabilities.  The specific attack exploited the fact that DEFLATE RFC1951 removes redundancy based on prefix matching.  This permitted the attacker to confirm guesses a character at a time, reducing an exponential-time attack into a linear-time attack. 7.2. QPACK mitigates but does not completely prevent attacks modeled on CRIME CRIME by forcing a guess to match an entire header field value, rather than individual characters.  An attacker can only learn whether a guess is correct or not, so is reduced to a brute force guess for the header field values. The viability of recovering specific header field values therefore depends on the entropy of values.  As a result, values with high entropy are unlikely to be recovered successfully.  However, values with low entropy remain vulnerable. Attacks of this nature are possible any time that two mutually distrustful entities control requests or responses that are placed onto a single HTTP/3 connection.  If the shared QPACK compressor permits one entity to add entries to the dynamic table, and the other to access those entries, then the state of the table can be learned. Having requests or responses from mutually distrustful entities occurs when an intermediary either: sends requests from multiple clients on a single connection toward an origin server, or takes responses from multiple origin servers and places them on a shared connection toward a client. Web browsers also need to assume that requests made on the same connection by different web origins RFC6454 are made by mutually distrustful entities. 7.3. Users of HTTP that require confidentiality for header fields can use values with entropy sufficient to make guessing infeasible.  However, this is impractical as a general solution because it forces all users of HTTP to take steps to mitigate attacks.  It would impose new constraints on how HTTP is used. Rather than impose constraints on users of HTTP, an implementation of QPACK can instead constrain how compression is applied in order to limit the potential for dynamic table probing. An ideal solution segregates access to the dynamic table based on the entity that is constructing header fields.  Header field values that are added to the table are attributed to an entity, and only the entity that created a particular value can extract that value. To improve compression performance of this option, certain entries might be tagged as being public.  For example, a web browser might make the values of the Accept-Encoding header field available in all requests. An encoder without good knowledge of the provenance of header fields might instead introduce a penalty for a header field with many different values, such that a large number of attempts to guess a header field value results in the header field not being compared to the dynamic table entries in future messages, effectively preventing further guesses. Simply removing entries corresponding to the header field from the dynamic table can be ineffectual if the attacker has a reliable way of causing values to be reinstalled.  For example, a request to load an image in a web browser typically includes the Cookie header field (a potentially highly valued target for this sort of attack), and web sites can easily force an image to be loaded, thereby refreshing the entry in the dynamic table. This response might be made inversely proportional to the length of the header field value.  Disabling access to the dynamic table for a header field might occur for shorter values more quickly or with higher probability than for longer values. 7.4. Implementations can also choose to protect sensitive header fields by not compressing them and instead encoding their value as literals. Refusing to insert a header field into the dynamic table is only effective if doing so is avoided on all hops.  The never indexed literal bit (see literal-name-reference) can be used to signal to intermediaries that a particular value was intentionally sent as a literal. An intermediary MUST NOT re-encode a value that uses a literal representation with the 'N' bit set with another representation that would index it.  If QPACK is used for re-encoding, a literal representation with the 'N' bit set MUST be used.  If HPACK is used for re-encoding, the never indexed literal representation (see Section 6.2.3 of RFC7541) MUST be used. The choice to mark that a header field should never be indexed depends on several factors.  Since QPACK doesn't protect against guessing an entire header field value, short or low-entropy values are more readily recovered by an adversary.  Therefore, an encoder might choose not to index values with low entropy. An encoder might also choose not to index values for header fields that are considered to be highly valuable or sensitive to recovery, such as the Cookie or Authorization header fields. On the contrary, an encoder might prefer indexing values for header fields that have little or no value if they were exposed.  For instance, a User-Agent header field does not commonly vary between requests and is sent to any server.  In that case, confirmation that a particular User-Agent value has been used provides little value. Note that these criteria for deciding to use a never indexed literal representation will evolve over time as new attacks are discovered. 7.5. There is no currently known attack against a static Huffman encoding. A study has shown that using a static Huffman encoding table created an information leakage, however this same study concluded that an attacker could not take advantage of this information leakage to recover any meaningful amount of information (see PETAL). 7.6. An attacker can try to cause an endpoint to exhaust its memory. QPACK is designed to limit both the peak and stable amounts of memory allocated by an endpoint. The amount of memory used by the encoder is limited by the protocol using QPACK through the definition of the maximum size of the dynamic table, and the maximum number of blocking streams.  In HTTP/3, these values are controlled by the decoder through the settings parameters SETTINGS_QPACK_MAX_TABLE_CAPACITY and SETTINGS_QPACK_BLOCKED_STREAMS, respectively (see maximum-dynamic-table-capacity and blocked- streams).  The limit on the size of the dynamic table takes into account the size of the data stored in the dynamic table, plus a small allowance for overhead.  The limit on the number of blocked streams is only a proxy for the maximum amount of memory required by the decoder.  The actual maximum amount of memory will depend on how much memory the decoder uses to track each blocked stream. A decoder can limit the amount of state memory used for the dynamic table by setting an appropriate value for the maximum size of the dynamic table.  In HTTP/3, this is realized by setting an appropriate value for the SETTINGS_QPACK_MAX_TABLE_CAPACITY parameter.  An encoder can limit the amount of state memory it uses by signaling a lower dynamic table size than the decoder allows (see eviction). A decoder can limit the amount of state memory used for blocked streams by setting an appropriate value for the maximum number of blocked streams.  In HTTP/3, this is realized by setting an appropriate value for the QPACK_BLOCKED_STREAMS parameter.  An encoder can limit the amount of state memory by only using as many blocked streams as it wishes to support; no signaling to the decoder is requred. The amount of temporary memory consumed by an encoder or decoder can be limited by processing header fields sequentially.  A decoder implementation does not need to retain a complete list of header fields while decoding a header block.  An encoder implementation does not need to retain a complete list of header fields while encoding a header block if it is using a single-pass algorithm.  Note that it might be necessary for an application to retain a complete header list for other reasons; even if QPACK does not force this to occur, application constraints might make this necessary. </ins> While the negotiated limit on the dynamic table size accounts for much of the memory that can be consumed by a QPACK implementation,


new streams, reading only from the encoder stream, or closing the connection.  8. 8.1.

new streams, reading only from the encoder stream, or closing the connection. <ins> 7.7. An implementation of QPACK needs to ensure that large values for integers, long encoding for integers, or long string literals do not create security weaknesses. An implementation has to set a limit for the values it accepts for integers, as well as for the encoded length (see prefixed-integers). In the same way, it has to set a limit to the length it accepts for string literals (see string-literals). </ins> 8. 8.1.


would be too large to fit in a packet, however receivers MAY also limit ACK frame size further to preserve space for other frames. <del> When discarding unacknowledged ACK Ranges, a receiver MUST retain the largest received packet number.  A receiver SHOULD retain ACK Ranges containing newly received packets or higher-numbered packets. </del>  A receiver that sends only non-ack-eliciting packets, such as ACK frames, might not receive an acknowledgement for a long period of

This explains what needs to be kept and why. Specifically, you need to keep ranges unless you have other means of ensuring that you don't accept packets from those ranges again. You also need to keep the largest acknowledged so that you can get a packet number from subsequent packets. This also recommends that ACK frames include the largest acknowledged always. That is primarily to ensure that ECN works properly, and even there, you only disable ECN if you get some weird reordering, so it's probably not a big deal if you don't follow this recommendation. The issue is marked design, but the resolution here is basically a restatement of other text. I think we can run this through the design process, but it isn't really worth flagging this in a change log in my opinion.
NAME you should probably look at this. I have marked the associated issue anyway, but I'd appreciate another review.
In , NAME notes that added this "MUST": My understanding is that this prevents the largest acknowledged packet from going backwards. We use the monotonic increase of the Largest Acknowledged field in two ways: In ECN validation, this is used to filter out ACK frames that might have arrived out of order, which might have lower counts legitimately. In key updates, where the value of the largest acknowledged is used to drive the change to new keys and to prevent use of old keys after an update. In my view, having ECN validation fail is quite unfortunate, but likely workable. On the other hand, uncertainty about largest acknowledged could introduce instability in the key update process. If an endpoint's conception of what the value is changes, I doubt that we'll see extra key updates (I think that the AEAD would protect us from that), but we might leave open the possibility of interleaving of packets from two different key phases. I would suggest that we retain this "MUST".
For out of order ACK frames, you can use the packet number the ACK arrives in. I realize that we have tried to allow frames to be retransmitted, but I think retransmitting ACK frames without modifying the ACK delay/etc is so harmful we should probably prohibit that one frame from being retransmitted, since it creates a bunch of edge cases. Or we could let ECN validation could fail, but that'd be a bit sad. I'm confused about what the issue with key updates is? The largest acked you've ever seen doesn't change, just the largest acked from the last ACK frame. When determining the size of the packet number, we say: "The sender MUST use a packet number size able to represent more than twice as large a range than the difference between the largest acknowledged packet and packet number being sent." I'd rather not make this a MUST unless we have a good reason. There are valid use cases that become more complex with this constraint.
I really want to understand the use cases that you think that this might break. Because key updates very much depend on tracking largest acknowledged. The key updates case has a fairly simple example. Say you receive packet 10 in key phase 1. Then 11 in key phase 2. That's valid. But if you ACK those and then only remember up to packet 8, packet 9 can use key phase 2 and you won't know that it is invalid.
I'd like to suggest tracking largest acknowledged from forcing all ACK frames to contain the largest acknowledged and have that value never decrease. Given you need to track largest acknowledged locally(ie: not only in the in-flight ACK frame), I don't think there are any problems there. I have two cases in mind where the largest acked could validly decrease: 1) A receiver that tracks a limited number of ACK blocks and the oldest block happens to be the one containing largest acked. 2) If an application was reordering tolerant(ie: from URL) then two different senders on a single 'connection' may be sending packets with disparate packet numbers, causing what appears to be very large scale reordering. In this case, it'd be valuable to discard the older ACK ranges even if they contained the largest acked. Maybe the first is an edge case we don't care about in a use case we aren't optimizing for and the latter is solvable with careful issuing of packet number ranges. Even so, I'm not sure what the motivation is for this change? I guess the ECN case could be, but using largest acked as a proxy for retransmitted ACKs is a very bad design in my opinion.
I don't think we really need to cater for retransmitted ACK frames in any way. What I'm concerned about is reordering. Yes, using the increase of largest acknowledged as a proxy for strict ordering is a little weird, but the assumption is that frame handling is distinct from packet handling and packet numbers are not available when frames are handled. (I think that's the principle we agreed on; that's why it's OK to use packet numbers in key updates, for example, something that I can confirm is borne out by implementation experience.) The case you describe here has the appearance of exactly that type of reordering if the two senders also generate their own ACK frames (though this seems very strange). But I think that the consequences of that sort of design are not our responsibility to deal with.
I don't recall on us agreeing that packet numbers weren't available when frames are handled, can you clarify why that property is important?
I agree with NAME on the principle of not requiring the enclosing packet's packet number during frame handling, and we use sequence numbers within frames to help with reordering. This was at least true in an earlier version of NCID frames, and we had discussed this principle then. This is also why the ACK_FREQUENCY frame in URL includes a sequence number. That said, I don't understand the issue with key updates here. Largest acked so far is a connection-level state variable that can be used for key updates. I suspect that I'm not understanding the example scenario. NAME can you elaborate on your key update example? NAME Is your example motivated by a receiver that does not remember the largest acked or one that does not wish to ack anything but the packet that was just received?
I was under the impression that the argument was that largest acknowledged was not being tracked, but Ian made it clear that the concern was about not signaling it. That removes key updates as a concern. If remains unclear, then I can't really do better.
NAME my example was motivated by either case, but primarily not wanting to ACK anything besides what was just received.
Perhaps it's a naive model, but I had envisioned an implementation remembering a chain of ranges of packet numbers it had received or not. The implementation would save space by forgetting the earliest packet numbers when the chain got too long, and would construct an ACK frame out of the latest set of these ranges it had room for. In that mental model, at least, there's not really room for the idea that you might drop the largest-acked frame number from your memory because it's too old -- it's always at the most-recent end of your chain, even if you currently happen to be receiving packets that fill in the middle for some implementation-specific reason. (Implementation-specific to the peer, which you obviously can't know about or code for!)
That matches my model for this. It is also likely the case that the next packet you receive is largest+1, so there is no real advantage to forgetting the largest acknowledged. Not sending largest acknowledged is slightly different though. I can understand how an implementation might choose to send only ranges that have recently changed. My contention is that this is not a good choice and we should recommend against that.
NAME to produce some text here that separates retention policy (MUST) from signaling policy (SHOULD).

would be too large to fit in a packet, however receivers MAY also limit ACK frame size further to preserve space for other frames.  <ins> A receiver MUST retain an ACK Range unless it can ensure that it will not subsequently accept packets with numbers in that range. Maintaining a minimum packet number that increases as ranges are discarded is one way to achieve this with minimal state. Receivers can discard all ACK Ranges, but they MUST retain the largest packet number that has been successfully processed as that is used to recover packet numbers from subsequent packets; see packet- encoding. A receiver SHOULD include an ACK Range containing the largest received packet number in every ACK frame.  The Largest Acknowledged field is used in ECN validation at a sender and including a lower value than what was included in a previous ACK frame could cause ECN to be unnecessarily disabled; see ecn-validation. </ins> A receiver that sends only non-ack-eliciting packets, such as ACK frames, might not receive an acknowledgement for a long period of