File: //proc/self/root/proc/thread-self/root/usr/share/doc/varnish-6.0.3/html/phk/spdy.html
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
"http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<meta http-equiv="X-UA-Compatible" content="IE=Edge" />
<meta http-equiv="Content-Type" content="text/html; charset=utf-8" />
<title>What SPDY did to my summer vacation — Varnish version 6.0.3 documentation</title>
<link rel="stylesheet" href="../_static/classic.css" type="text/css" />
<link rel="stylesheet" href="../_static/pygments.css" type="text/css" />
<script type="text/javascript" id="documentation_options" data-url_root="../" src="../_static/documentation_options.js"></script>
<script type="text/javascript" src="../_static/jquery.js"></script>
<script type="text/javascript" src="../_static/underscore.js"></script>
<script type="text/javascript" src="../_static/doctools.js"></script>
<link rel="index" title="Index" href="../genindex.html" />
<link rel="search" title="Search" href="../search.html" />
<link rel="next" title="Why HTTP/2.0 does not seem interesting" href="http20.html" />
<link rel="prev" title="Wanton destruction for the greater good" href="wanton_destruction.html" />
</head><body>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
accesskey="I">index</a></li>
<li class="right" >
<a href="http20.html" title="Why HTTP/2.0 does not seem interesting"
accesskey="N">next</a> |</li>
<li class="right" >
<a href="wanton_destruction.html" title="Wanton destruction for the greater good"
accesskey="P">previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">Varnish version 6.0.3 documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="index.html" accesskey="U">Poul-Hennings random outbursts</a> »</li>
</ul>
</div>
<div class="document">
<div class="documentwrapper">
<div class="bodywrapper">
<div class="body" role="main">
<div class="section" id="what-spdy-did-to-my-summer-vacation">
<span id="phk-spdy"></span><h1>What SPDY did to my summer vacation<a class="headerlink" href="#what-spdy-did-to-my-summer-vacation" title="Permalink to this headline">¶</a></h1>
<p>It’s dawning on me that I’m sort of the hipster of hipsters, in the sense
that I tend to do things far before other people do, but totally fail to
communicate what’s going on out there in the future, and thus by the
time the “real hipsters” catch up, I’m already somewhere different and
more interesting.</p>
<p>My one lucky break was the <a class="reference external" href="http://bikeshed.org/">bikeshed email</a> where
I actually did sit down and compose some of my thoughts, thus firmly
sticking a stick in the ground as one of the first to seriously think
about how you organize open source collaborations.</p>
<p>I mention this because what I am going to write probably seems very
unimportant for most of the Varnish users right now, but down the road,
three, five or maybe even ten years ahead, I think it will become important.</p>
<p>Feel free to not read it until then.</p>
<div class="section" id="the-evolution-of-varnish">
<h2>The evolution of Varnish<a class="headerlink" href="#the-evolution-of-varnish" title="Permalink to this headline">¶</a></h2>
<p>When we started out, seven years ago, our only and entire goal was to build
a server-side cache better than squid. That we did.</p>
<p>Since then we have added stuff to Varnish (ESI:includes, gzip support,
VMODS) and I’m staring at streaming and conditional backend fetches right
now.</p>
<p>Varnish is a bit more than a web-cache now, but it is still, basically,
a layer of polish you put in front of your webserver to get it to
look and work better.</p>
<p>Google’s experiments with SPDY have forced a HTTP/2.0 effort into motion,
but if past performance is any indication, that is not something we have
to really worry about for a number of years. The IETF WG has still to
manage to “clarify” RFC2616 which defines HTTP/1.1, and to say there
is anything even remotely resembling consensus behind SPDY would be a
downright lie.</p>
<p>RFC2616 is from June 1999, which, to me, means that we should look at
2035 when we design HTTP/2.0, and predicting things is well known to
be hard, in particular with respect to the future.</p>
<p>So what’s a Varnish architect to do?</p>
<p>What I did this summer vacation, was to think a lot about how Varnish
can be architected to cope with the kind of changes SPDY and maybe HTTP/2.0
drag in: Pipelining, multiplexing, etc., without committing us to one
particular path of science fiction about life in 2035.</p>
<p>Profound insights often sound incredibly simplistic, bordering
trivial, until you consider the full ramifications. The implementation
of “Do Not Kill” in current law is surprisingly voluminous. (If
you don’t think so, you probably forgot to #include the Vienna
Treaty and the convention about chemical and biological weapons.)</p>
<p>So my insight about Varnish, that it has to become a socket-wrench-like
toolchest for doing things with HTTP traffic, will probably elicit a lot
of “duh!” reactions, until people, including me, understand the
ramifications more fully.</p>
</div>
<div class="section" id="things-you-cannot-do-with-varnish-today">
<h2>Things you cannot do with Varnish today<a class="headerlink" href="#things-you-cannot-do-with-varnish-today" title="Permalink to this headline">¶</a></h2>
<p>As much as Varnish can be bent, tweaked and coaxed into doing today,
there are things you cannot do, or at least things which are very
hard and very inefficient to do with Varnish.</p>
<p>For instance we consider “a transaction” something that starts with
a request from a client, and involves zero or more backend fetches
of finite sized data elements.</p>
<p>That is not how the future looks.</p>
<p>For instance one of the things SPDY has tried out is “server push”,
where you fetch index.html and the webserver says “you’ll also want
main.css and cat.gif then” and pushes those objects on the client,
to save the round-trip times wasted waiting for the client to ask
for them.</p>
<p>Today, something like that is impossible in Varnish, since objects
are independent and you can only look up one at a time.</p>
<p>I already can hear some of you amazing VCL wizards say “Well,
if you inline-C grab a refcount, then restart and …” but let’s
be honest, that’s not how it should look.</p>
<p>You should be able to do something like:</p>
<div class="highlight-default notranslate"><div class="highlight"><pre><span></span><span class="k">if</span> <span class="p">(</span><span class="n">req</span><span class="o">.</span><span class="n">proto</span> <span class="o">==</span> <span class="s2">"SPDY"</span> <span class="o">&&</span> <span class="n">req</span><span class="o">.</span><span class="n">url</span> <span class="o">~</span> <span class="s2">"index.html"</span><span class="p">)</span> <span class="p">{</span>
<span class="n">req</span><span class="o">.</span><span class="n">obj1</span> <span class="o">=</span> <span class="n">lookup</span><span class="p">(</span><span class="n">backend1</span><span class="p">,</span> <span class="s2">"/main.css"</span><span class="p">)</span>
<span class="k">if</span> <span class="p">(</span><span class="n">req</span><span class="o">.</span><span class="n">obj1</span><span class="o">.</span><span class="n">status</span> <span class="o">==</span> <span class="mi">200</span><span class="p">)</span> <span class="p">{</span>
<span class="n">sess</span><span class="o">.</span><span class="n">push</span><span class="p">(</span><span class="n">req</span><span class="o">.</span><span class="n">obj1</span><span class="p">,</span> <span class="n">bla</span><span class="p">,</span> <span class="n">bla</span><span class="p">,</span> <span class="n">bla</span><span class="p">);</span>
<span class="p">}</span>
<span class="n">req</span><span class="o">.</span><span class="n">obj2</span> <span class="o">=</span> <span class="n">lookup</span><span class="p">(</span><span class="n">backend1</span><span class="p">,</span> <span class="s2">"/cat.gif"</span><span class="p">)</span>
<span class="k">if</span> <span class="p">(</span><span class="n">req</span><span class="o">.</span><span class="n">obj1</span><span class="o">.</span><span class="n">status</span> <span class="o">==</span> <span class="mi">200</span><span class="p">)</span> <span class="p">{</span>
<span class="n">sess</span><span class="o">.</span><span class="n">push</span><span class="p">(</span><span class="n">req</span><span class="o">.</span><span class="n">obj2</span><span class="p">,</span> <span class="n">bla</span><span class="p">,</span> <span class="n">bla</span><span class="p">,</span> <span class="n">bla</span><span class="p">);</span>
<span class="p">}</span>
<span class="p">}</span>
</pre></div>
</div>
<p>And doing that is not really <em>that</em> hard, I think. We just need
to keep track of all the objects we instantiate and make sure they
disappear and die when nobody is using them any more.</p>
<p>A lot of the assumptions we made back in 2006 are no longer
valid under such an architecture, but those same assumptions are
what gives Varnish such astonishing performance, so just replacing
them with standard CS-textbook solutions like “garbage collection”
would make Varnish lose a lot of its lustre.</p>
<p>As some of you know, there is a lot of modularity hidden inside
Varnish but not quite released for public use in VCL. Much of what
is going to happen will be polishing up and documenting that
modularity and releasing it for you guys to have fun with, so it
is not like we are starting from scratch or anything.</p>
<p>But some of that modularity stands on foundations which are no longer
firm; for instance, the initiating request exists for the full duration of
a backend fetch.</p>
<p>Those will take some work to fix.</p>
<p>But, before you start to think I have a grand plan or even a clear-cut
road map, I’d better make it absolutely clear that is not the case:
I perceive some weird shapes in the fog of the future and I’ll aim
in that direction and either they are the doorways I suspect
or they are trapdoors to tar-pits, time will show.</p>
<p>I’m going to be making a lot of changes and things that used to be
will no longer be as they used to be, but I think they will be better
in the long run, so please bear with me, if your favourite detail
of how Varnish works changes.</p>
</div>
<div class="section" id="varnish-is-not-speedy-varnish-is-fast">
<h2>Varnish is not speedy, Varnish is fast!<a class="headerlink" href="#varnish-is-not-speedy-varnish-is-fast" title="Permalink to this headline">¶</a></h2>
<p>As I said I’m not a fan of SPDY and I sincerely hope that no bit of
the current proposal survives unchallenged in whatever HTTP/2.0 standard
emerges down the road.</p>
<p>But I do want to thank the people behind that mess, not for the
mess, but for having provoked me to spend some summertime thinking
hard about what it is that I’m trying to do with Varnish and what
problem Varnish is here to solve.</p>
<p>This is going to be FUN!</p>
<p>Poul-Henning 2012-09-14</p>
<p>Author of Varnish</p>
<p>PS: See you at <a class="reference external" href="https://www.varnish-cache.org/vug6">VUG6</a> where I plan
to talk more about this.</p>
</div>
</div>
</div>
</div>
</div>
<div class="sphinxsidebar" role="navigation" aria-label="main navigation">
<div class="sphinxsidebarwrapper">
<h3><a href="../index.html">Table Of Contents</a></h3>
<ul>
<li><a class="reference internal" href="#">What SPDY did to my summer vacation</a><ul>
<li><a class="reference internal" href="#the-evolution-of-varnish">The evolution of Varnish</a></li>
<li><a class="reference internal" href="#things-you-cannot-do-with-varnish-today">Things you cannot do with Varnish today</a></li>
<li><a class="reference internal" href="#varnish-is-not-speedy-varnish-is-fast">Varnish is not speedy, Varnish is fast!</a></li>
</ul>
</li>
</ul>
<h4>Previous topic</h4>
<p class="topless"><a href="wanton_destruction.html"
title="previous chapter">Wanton destruction for the greater good</a></p>
<h4>Next topic</h4>
<p class="topless"><a href="http20.html"
title="next chapter">Why HTTP/2.0 does not seem interesting</a></p>
<div role="note" aria-label="source link">
<h3>This Page</h3>
<ul class="this-page-menu">
<li><a href="../_sources/phk/spdy.rst.txt"
rel="nofollow">Show Source</a></li>
</ul>
</div>
<div id="searchbox" style="display: none" role="search">
<h3>Quick search</h3>
<div class="searchformwrapper">
<form class="search" action="../search.html" method="get">
<input type="text" name="q" />
<input type="submit" value="Go" />
<input type="hidden" name="check_keywords" value="yes" />
<input type="hidden" name="area" value="default" />
</form>
</div>
</div>
<script type="text/javascript">$('#searchbox').show(0);</script>
</div>
</div>
<div class="clearer"></div>
</div>
<div class="related" role="navigation" aria-label="related navigation">
<h3>Navigation</h3>
<ul>
<li class="right" style="margin-right: 10px">
<a href="../genindex.html" title="General Index"
>index</a></li>
<li class="right" >
<a href="http20.html" title="Why HTTP/2.0 does not seem interesting"
>next</a> |</li>
<li class="right" >
<a href="wanton_destruction.html" title="Wanton destruction for the greater good"
>previous</a> |</li>
<li class="nav-item nav-item-0"><a href="../index.html">Varnish version 6.0.3 documentation</a> »</li>
<li class="nav-item nav-item-1"><a href="index.html" >Poul-Hennings random outbursts</a> »</li>
</ul>
</div>
<div class="footer" role="contentinfo">
© Copyright 2010-2014, Varnish Software AS.
Created using <a href="http://sphinx-doc.org/">Sphinx</a> 1.7.6.
</div>
</body>
</html>