What to do about HPSS protocol backwards compatibility for faster push?

[John Arbash Meinel]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Andrew Bennetts wrote:
> John Arbash Meinel wrote:
> [...]
>> Andrew Bennetts wrote:
> [...]
>> 6) Have large requests use a streaming protocol (chunked transfer encoding) so
>> they can fail early rather than after streaming all 10 MB across the wire.
>> Obviously this requires getting chunked/streaming working.
> 
> This isn't as easy as it sounds.  We currently assume that a request won't get a
> response until it is fully sent, and changing this will make working over HTTP
> more difficult.  We'd could map it onto the “Expect: 100-continue”/“100
> Continue” feature of HTTP/1.1, but there are pitfalls there (like the RFC says
> that clients SHOULD NOT wait for an indefinite period for a 100 Continue/417
> Expectation failed before sending the request body because older servers and
> proxies might not transmit them.
> 
> Really, this is extremely close to just sending an explicit “is request X
> supported?” request just before the actual request.  In terms of round trips on
> the wire it's identical.
> 
> -Andrew.

Except you can send some bytes with that first one. But yes, there is a latency
round trip. I'm not sure if you can mask some of the latency by tuning the size
of the first packet.

If you assume that the server can reject it as soon as it reads the request
name, then you just need to send enough bytes to mask the round trip latency.
eg. if you have a 100ms ping and can send at 10KB/s, you need to send 1KB of
data. And then you will either have a rejection packet waiting for you, or you
could assume you could keep sending.

More practically, you could at least save 1/2 the latency by sending a decent
chunk of data up. (Since you should at least wait for the server to say OK
before sending more.)

I think Michael's suggestion is actually quite good here. And depending on what
the request expects, could be fairly easily done by just the client.

John
=:->

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.7 (Darwin)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iD8DBQFHFsqMJdeBCYSNAAMRAj77AJwOGUEGt2t9OMrbs2euban8BoElFACfSXB2
EpXtT6cUfLrkG7XZBVo1tDc=
=+8CU
-----END PGP SIGNATURE-----