Here are my raw, unedited notes from Mark Nottingham's QCon San Francisco 2008 talk.
- Protocols are hard, and if you don't understand them you'll repeat the mistakes of the past.
- Goal is to teach what HTTP can do, what implementations can't
- HTTP 1.1 contain damage from prior versions
- HTTP-NG underway
- In the old days, services were not thought of, only browswer/WebDAV scenarios
- IETF now interested in supporting RESTful Web Services
- HTTP 2.0 didn't happen, but SOAP did
- HTTP is a protocol construction toolkit (e.g. AtomPub)
- Lots of new implementations (servers, clients, frameworks, APIs)
- HTTPbis to clarify HTTP 1.1 in light of new innovative uses, maintain interoperability. Rationale: HTTP outlives its creators
-HTTPBis splits out HTTP spec into messaging, semantics, payload, conditionals, ranges, caching, and authentication
- HTTPbis fixes many issues in HTTP spec, from typos to prose, to ABNF, ETags on PUT, whitespace issues, cache key
- HTTPbis is "six months" away
- Implementation explosion:
- clients, XmlHttpRequest, Flash.
- servers: IIS, Apache, your fridge. Web servers ubiquitous nowadays
- lots more intermediaries - 20-30% of Web traffic goes through a proxy.
- caching showing up in clients
- Most everything today is HTTP 1.1.
- Core methods: GET, POST everywhere. PUT DELETE getting better support (sometimes intermediaries block).
- Recall X-Http-Method for tunnelling methods over POST
- Limitation of W3C languages for understanding verbs
- OPTIONS is problematic for clients and servers, is not cacheable
- Extensions methods not allowed by many clients - implementation support problems, intermediaries may block
- Use the standard methods, don't extend methods, use POST instead
- Question: HTTP 1.1 allows extended methods? Answer: yes.
- URI length limits - some clients as short as 256 chars. Most clients allow really big URIs. Intermediaries vary - typically 4-8Kb.
- Results in people tunnelling queries through POST (no limit on entity body size), killing cacheability - may not be a problem
- Some frameworks embody this ant-ipattern
- HTTPbis to recommend URIs at 8Kb
- Headers have some length limits (Squid: 20Kb in total)
- Header line continuations aren't honoured - HTTPbis cans them
- No trailers
- Redirect supported for POST, but not PUT or DELETE. 307 well supported, but bot 301 or 302
- Basic conformance exists for caching (max-age, non-cache, etc)
- Invalidation isn't widely implementated (POSTing through a cache invalidates any cached representations)
- Some clients (curl) don't cache by default, opportunity cost incurred
- Connection handling - limit browsers to 2 concurrent connections to a server. Being fixed by HTTPbis - no predefined limits, just behave responsibly.
- Pipelining - only Opera and a handful of libraries understand it, intermediaries won't forward, few servers fail on it.
- Pipelining risks out of order responses - catastrophic
- SVN is a common use case for pipelining
- Fixing pipelining addresses lots of roundtrips problem
- Partial content, driven mostly by accessing PDF bytes. Some caches don't support Content-Range, some APIs (e.g. flash URL) support it, but it may not be used. Result: encode the range as query params in the URI - kills caching because of duplication (non-overlapping streams)
- Cookies - "I hate cookies" They're crap and not specified well. RFC2109 does not reflect current practice; RFC2965 lacks implementation support (Opera only). Need special libraries to parse cookies.
- HTTP may become test-centric - ability to test for conformant implementations on the wire; open source could be used to test clients and services
- Authentication - basic is insecure, digest is not interoperable. Some new requirements not addressed (e.g. phishing, delegation). OAuth at BoF level within IETF - WG to come?
- Better transport - integrity not guaranteed (need checksum), binding to SCTP (telecoms heritage) better for lossy networks - arbitrary streams over single SCTP connection - multiplex 10,000 requests over 1 connection - solves head of line blocking; stop managing TCP, use a stream abstraction that just works. May help in 10 years time, or sooner for internal networks
- PATCH becoming interesting, as a diff-based update. Don't use partial PUT, it's dangerous.
- Question: What to do until PATCH comes round? Answer: Use POST instead until PATCH is widespread.
- Prefer header - let's a client state what it wants - e.g. give me 204 rather than 200 plus entity body
- Link header - advertising links in HTTP headers (e.g. next, previous). Generalise it in the headers, rather than burying it in the representations then intermediaries can process it - no longer Atom or HTML specific
- Question: Any link with XML schema? Answer: No. A little part of MNot dies each time he sees an XML schema
- Caching refinements - allow caches to serve a stale response while refreshing asynchronously; allow serving a slightly stale response if origin server is dead
- Out of band change monitoring - cache channels, spec published. More useful than just for caching. Next step to generalise
- Link headers useful for cache invalidation
- HTTP software heading towards higher-level RESTful abstractions: webmachine, better coverage of features in software generally - raise the quality bar for implementations; opportunity for intermediary construction kit?
- O2.0 - Open 2.0. Many pragmatic things (OAuth, OpenID, Comet, XmlHttpRequest, etc), lowest common denominator. Hard to push into the overall architecture - high opportunity cost (work in bounded cases, break other parts of the Web).
- IETF and OAuth love fest
- Summary: Varying quality of implementations, lots of workarounds. Devs take the easiest path. HTTPbis should improve matters.
- Question: HTTPbis is open yet? Answer: Lot of activity until around 6 months ago. Work being resumed - traffic on list increasing again.
- Question: Timeframe for HTTPbis? Answer: Not much to change, only clarifications.
- Question: POST for everything, why even PUT? Answer: Server can retry PUTs automatically, as can clients. PUT benefits over POST less clear than GET/DELETE. Sometimes it does not make sense to use 15 PUTs instead of one POST. Followup: CouchDB makes good use of PUT. Answer followup: Generic storage of bytes is fine for PUT, more specific semantics is trickier.
- URI and method together should not be cacheable - URI only is better.
- Question: is there a compatibility check for HTTPbis. Answer: Some effort, part time on MNot's behalf. Big broad space to test.