To be blunt, this makes absolutely no sense. You tell me so, and the next day, I tell you to check it out again. You grumble, and get back to the curl:
$ curl -i http://example.com/profile
HTTP/1.1 200 OK
Content-Type: text/plain
Content-Length: 548
The Example.com API
===================
Example.com provides access to our blog through an API.
In the API, you'll see two major things of interest: `wtl` and `grobb34s`.
## wtl
The value provided under the `wtl` key is the time the latest blog post
was posted, in "%Y-%m-%d %H:%M:%S %z" format. This value is then Base64
encoded.
## grobb34s
The `grobb34s` key will hold an array of blog posts. These posts are
represented by a JSON object with two keys. `flog` has the title, suitable for
display, and `zilch` contains a link to the post.
Oh. You don't care about some blog with such terrible titles, so you go about your day.
self-descriptive-ness
You may consider this a highly contrived example, but it's not as contrived as you think. In Fielding's thesis, he describes a 'self-described messages' constraint, which means that any API call should be able to be understood alone, without additional context.
In HTTP, this information is generally provided via the Content-Type header. It describes a media type (such as application/json in this example) that describes the rules for processing the response. When writing a HTTP client, you fetch the response, check the Content-Type, and then invoke the correct parser based on the type provided.
But many times are very general. Take application/json, for example. It says nothing about blog posts. A user-agent that only knows about application/json is very much like you as a human seeing gibberish keys and values; it's not smart enough to make assumptions based on the text of keys and values it may see in JSON. However, with the added context of a profile, we have enough information to make sense of this strange API. And any user-agent that sees a profile that it recognizes can act on those new semantics, too.
JSON API
This is one of the reasons that we're working on JSON API, a standard media type for APIs that use JSON. Generic JSON has no standard semantics that are useful to API authors, since it's a very generic format. By declaring a new media type with common features to many APIs, we can write generic tools that handle the 80% of cases that come up during API development.
And if the generic case doesn't suit your requirements, you can always extend it with profiles. In fact, it'd be nice if many people just added a Link header to their existing API documentation; that'd start alleviating this problem.
If you enjoyed this article, you might want to check out my in-progress book on building APIs that respect standards and HTTP, Designing Hypermedia APIs. This post also serves as its first blog post.
I spent six weeks of last December/January in my hometown. It was the first Christmas without my father, and it was cheap to stay, so I went back. It was my first time back in Pittsburgh since I'd moved away from the only place I'd ever lived.
The trip was incredibly emotional for many reasons, but one of the best was when I decided to walk down the street to get a pizza. I had to wait ten minutes for it to cook, and so I decided to open up my web browser. Then I saw this:
Whoah.
You see, I've always had a complex relationship with _why. I had been dabbling in Ruby when he was still alive, and planned on dropping by this ART && CODE thing I'd heard about over at CMU, but I was out of town that weekend, so I shrugged and didn't think twice.
Then he disappeared.
I started looking into _why and what he was actually about. What I found was amazing, and you already know that part of the story. I thought Hackety Hack was a masterpiece, and decided to help maintain it. Unfortunately for me, nobody else did. I hadn't done a whole lot of open source work before, and now I was maintaining a big, important project by one of the most well-known Rubyists of all time. Whoah.
I cared about Hackety enough and felt inspired enough by _why that I actually quit my startup to spend more time working on it.
Here's the weird thing about someone disappearing: you can't ask them questions. I had so many things that I wanted to ask, so many unresolved questions, so much I wanted to know. It's really weird to take someone's life's work and make it your own. I spent a lot of that first year with crippling doubts about what I was doing, if _why would like it, if that even mattered, wondering why he'd abandoned something so important, why he'd abandoned us. And I felt terrible that I needed this from him. Why does he owe us all that? Who am I to impose?
So I marked my calendar for the 18th, and waited. That was yesterday.
It was… great. Just great. The stuff _why sent is basically everything I'd ever hoped. It's got so much buried inside. An IF game, commentary on Jobs vs. Gates, a very person struggle with identity, a comment that he likes what we've done… it's great. And it's finally brought me some closure.
If you truly want to understand technology today, then you should at least be familiar with the philosophy of Gilles Deleuze. Unfortunately for technologists, Deleuze is rooted firmly in a philosophical tradition and a writing style that they probably find opaque. In this blog series, I plan on explaining Deleuze’s philosophy in terms that programmers can understand. This is the third in the series. You can find the first here. Enjoy.
Deleuze and Guattari employ the notion of 'smooth space' quite a bit in A Thousand Plateaus. I saw this quote about it the other day:
never believe that a smooth space will suffice to save us
I've recently been giving a lot of thought to 'cultural hegemony'
and its applicability to the web, specifically along the lines of
Google. Basically, cultural hegemony is the idea that you don't just
have dominance through laws or economics, but through culture. So, for
example, Google submitted SPDY to become HTTP 2.0: it's going through
the standards process, it's 'open,' but it also gives Google a lot of
cultural leverage.
So there's this connection there between smooth space and open source, and the battle between the Nomads and the State Apparatus. We'll cover that second bit in a while, but for now: what is 'smooth space'?
The spatial metaphor
The first thing to understand about these concepts is that Deleuze and
Guattari is (I think of them as a singular entity…) a big fan of
spatial metaphors. Essentially, we can project our problem domain onto
a 2 or 3 (or more) dimensional space, and then use tools to analyze
that space. I was first introduced to this technique by Manuel de
Landa, who has a great lecture called “Deleuze and the use of the
genetic algorithm in archetecture” (video)(text). A quick rundown:
So, let's talk about the N queens problem. We have a chess board
with one space, and we place one queen on it:
(warning, lots of ASCII (+ unicode) art to follow)
.-.
|♕|
.-.
This board is legal. It's a solution to the '1 queen problem': one
queen, a 1x1 board. What about 2 queens? We have no solutions. (2 and
3 don't. :/ All other natural numbers do.) First we place a queen:
.---.
|♕| |
-----
| | |
.---.
… but there's no legal place for the second one. Bummer. We can
determine that there's no solution by using a simple brute force with
backtracking: place the queen in the upper right, then try to place
the second below it. That's not legal, so we backtrack and place it on
the right. That's not legal, so we backtrack and put it in the bottom
corner. Oh no! That didn't work, so let's move our first queen: now
it's in the bottom right, and we try to place the second in the upper
right: fail! So we backtrack….
As you can see, this is a lot of steps for a small board. Once you get
up to a 'real' chess board, there are 4,426,165,368 possible
placements, but only 92 solutions. A needle in a haystack! And we're
using a lot of steps to determine if there's even one possible
placement. We need something better.
One answer is genetic algorithms. So we can take our board's current
state and assign it a score. When we place a new queen, if it makes
our score better, we keep it, and if it doesn't, we lose it. Now we've
taken our 2x2 board and projected it onto a 2-axis linear space.
Imagine a plot where queens are on the x axis and score is on the y
axis:
So we want to keep going with our solution as long as it slopes up:
we've found that our answers are at the maximum of each curve. Yay
calculus! But here's the thing: this curve only has two possible
maxima: we will have more. And it might not be in 2-d space, it might
be in 5-d space. And we can only 'use math' to find the maximum if we
generate the entire curve, which seems computationally out of our
reach. So how do we find the maximum without generating the whole
curve? Genetic algorithms!
One such is 'simulated annealing.' Without getting too into it,
let's just say that there's a 'cooling factor' that controls how
tolerant we are of going back down a slope. So at first, we wildly go
all over the search space, but then, as we progress, we tighten up our
cooling factor, and we stop being so wild. Eventually, we'll arrive at
a solution that's very likely to be the true global maxima/minima.
Neat!
Discrete vs. Continuous
If we know that the enemy is open to attack, and also know that our men are
in a condition to attack, but are unaware that the nature of the ground makes
fighting impracticable, we have still gone only halfway towards victory.
Sun Tzu, “the Art of War”
Another interesting feature of this particular projection is that it
transforms our problem from a discrete problem into a continuous one.
One great tactic for when you're losing: change the battlefield. Our
genetic algorithm tool needs a continuous space to operate, but our
problem is that our chess board is discrete. What do I mean by this?
“discrete space” is one in which the points are separated from one
another in some way. As an example, 'integers' form a discrete
topology: there's a 'gap' between 1, 2, 3, and 4. You can see this by
drawing a number line:
<--(-2)--(-1)--(0)--(1)--(2)-->
The real numbers form a continuous topology rather than a discrete
one, there is no space between them. A 'real line':
<------------------------->
Our 'scoring' mechanism allows us to change the battlefield, it forms
a function (isomorphism) to convert our discrete topology into a
continuous one. We can now bring our continuous tooling to bear on a
problem that was previously inaccessible.
Striated vs Smooth Space
Military tactics are like unto water; for water in its natural course runs
away from high places and hastens downwards. So in war, the way is to avoid
what is strong and to strike at what is weak. Like water, taking the line of
least resistance. Water shapes its course according to the nature of the
ground over which it flows; the soldier works out his victory in relation to
the foe whom he is facing. Therefore, just as water retains no constant
shape, so in warfare there are no constant conditions.
Sun Tzu, “the Art of War”
Okay. NOW we're ready to talk about smooth and striated space. They
have a number of names for this concept, and the one that's most
direct from where we currently are is 'Riemann space / Euclidean
space'. Smooth -> Riemann, Euclidean -> Striated. Another isomorphism.
;)
Euclidean geometry is pretty much the foundation of a metric ton of
our math. It's what's now known as algebra and geometry: lots of
integers, discrete spaces. This cube starts at (1,2,4) and has a side
length of 5. Cartesian coordinates.
Along comes Gauss, who was incredibly intelligent. He had this
student named Riemann, who ran with the ideas Gauss had and
started Riemannian Geometry. Riemann's geometry is
non-Euclidean: if Euclid described lines, Riemann described
curves. Einstein would later base the theory of relativity on
Riemannian geometry.
So, who cares? Well, think about it this way: You're in a car, driving
down a straight road. It goes for miles, with no turns or other
streets. You're operating in a smooth space: you can easily go from
one place to another. Now, someone comes along and puts up stop signs
every block. You must (for now) stop at these signs. Now you're
operating in a striated space: your movement is restricted. Now you're
moving from point to point on a line. It's a very different
experience, and you also didn't have anything to do with the stop
signs….
According to D&G, 'the state machine' (one instance of which is the
State) attempts to take space that is smooth and striate it. This is
often a means towards imperialism, as the striation happens through
quantification. Take, for example, the sea: a smooth space, you can
impose latitude and longitude on top of it, simultaneously quantifying
and striating it. This allows you to navigate the waters, and conquer
the sea.
This is also the position that many web startups take: 'friendship'
was a smooth space before social networks came along. They quantified
and striated that space, turning “I have many friends” into “I have
200 friends and you have 100 friends.” This quantification allows for
commodification: now Facebook can sell ads. It's also why startups
don't often have a 'business model' at first: they first need to
conquer and striate their space before they find the commodity.
Sometimes you just hit dirt and go bust, sometimes you find a more
precious commodity, then refine and sell it.
'nomads' are entities which navigate and live in smooth spaces. Maybe
my twitter bio makes a bit more sense now. ;)
How do you create a smooth space?
You might remember Riemann from the 'Riemann sum' in Calc I. The
brilliance of the Riemann sum is that it first striates, then
re-smooths the space. You start off first with a curve, and then
approximate the area under it by dividing it into a number of
evenly-sized bits. This first bit was necessary in a world without
calculus: we didn't have the tooling or concepts to actually tackle
the real area, so we mapped that problem to one we did know how to
solve. As the number of bits goes up, and their width goes down, more
and more of the space under the curve is captured by our algorithm.
Finally, once we're able to take that first step away from (striated)
algebra and move into (smooth) calculus, we're able to move about on
the curve for reals. We're back to smooth space again.
This interplay between smooth and striated spaces often happens, and
there's really no part of our world today that's completely smooth or
entirely striated. D&G posit that the left should be attempting to
create as much smooth space as possible, and that capitalism is
constantly attempting to striate space. That said, smooth space is
necessary, but not sufficient: capitalism, in its hunger and lust for
acquisition and totalitization, has managed to navigate some smooth
spaces. I'll just quote straight from 'a thousand plateaus', 492. It's
a little hard, but this is getting long enough as is. ;)
Not only does the user as such tend to be an employee, but capitalism
operates less on a quantity of labor than by a complex qualitative process
bringing into play modes of transportation, urban models, the media, the
entertainment industries, ways of perceiving and feeling – every semiotic
system. It is though, at the outcome of the striation that capitalism was
able to carry to an unequaled point of perfection, circulating capital
necessarily recreated, reconstituted, a sort of smooths pace in which the
destiny of human beings is recast. … at the level of world capitalism, a
new smooth space is produced in which capital reaches its 'absolute' speed,
based on machinic components rather than the human component of labor. The
multinationals fabricate a kind of deterritorialized smooth space in which
points of occupation as well as poles of exchange become quite independent of
the classical paths to striation. .. the essential thing is instead the
distinction between striated capital and smooth capital, and the way in which
the former gives rise to the latter through complexes that cut across
territories and States, and even the different types of States.
The first part about 'users' is very much drawing a parallel to “If
you’re not paying for the product, you are the product.” When they
talk about the 'speed' of capital, think of HFT. Multinational
corporations have managed to overcome the striation that a State has
imposed. McDonalds is only partially affected by US laws. They operate
'outside' of states.
In this way, simply being smooth will not suffice to save us.
If you liked this blog post, you may enjoy “Philosophy in a time of software, a moderated Google Group to discuss the intersection of philosophy and technology. This post started out as an email to that list.
Security is hard. One of the skills any OSS maintainer needs is how to do
releases, and security releases are a special kind of release that needs
special git-fu to make it work.
I've created a repository so that you can practice this particular skill. It's called 'security_release_practice'.
The problem
'security_release_practice' provides a binary, omg_insecure, which has a security issue.
Basically, the super_secure_calculation method converts user input to a
symbol. Since Ruby does not garbage collect symbols, the longer you run
omg_insecure, the more memory it will use, until your computer runs out of
memory and the box grinds to a halt. Seems bad.
So, just fix super_secure_calculation and release, right? Well, here's the
problem: the last release of security_release_practice was 1.0.0. Since
then, we've had a new feature, and a backwards incompatible change with
super_secure_calculation. You can see the two commits
here.
This is a problem: if we fix the issue and release, people who are relying on
the + 5 behavior can't upgrade: they'll now be getting + 6. Also, the new
feature (another_new_calculation) may have conflicts or weirdness with their
code. That's bad! So what we really want is a relase that's exactly the
same as 1.0.0, but with the security fix applied.
Let's give that a shot.
The answer
If you think you're good with git, you can try this out right now. If you've
done it correctly, you should end up with the following:
A 1-0-stable branch
That branch should contain a new commit that fixes the issue
That branch should contain a new tag, v1.0.1 that fixes the issue
Master should have a backported version of the commit in #2.
The repository as it
exists has all of
this stuff, so check your work against it!
Practice!
If you don't know how to do this, or you get stuck, you've come to the
right place! Here's what you need to do:
First, some setup work. Fork the repository and clone it down. Or, just clone
mine, whatever:
$ git clone https://github.com/steveklabink/security_release_practice
$ cd security_release_practice
Next, since this repository has the backported fix involved, you need
to remove that commit:
$ git reset --hard HEAD~1
This basically backs our branch out by one commit. Now we're ready to go.
The first thing in actually doing the work is to check out the tag that
we last released from. In our case, that tag is v1.0.0. So let's do that
now:
$ git checkout v1.0.0
git will give you a message:
Note: checking out 'v1.0.0'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b new_branch_name
HEAD is now at 47a8bfb... Initial release.
We want to take git's advice: let's make a new branch. Since it's going to
be all our fixes for 1.0.x, let's call it 1-0-stable:
git checkout -b 1-0-stable
Now we have our stable branch. Awesome! Master is the work that will go into
1.1.x, and this branch will be for 1.0.x.
Next, we need to fix our bug. You need to remove this one line:
We don't even use that symbol, what a waste! Commit this, with a descriptive
message. You can find mine here.
Note the first few characters of the hash: mine is 168d5f756221.
Next, we need to release. Go ahead and increment the version number in
lib/security_release_practice/version.rb, commit that, and then try
rake install. Everything should work. Great! If you were actually releasing
this gem, you'd be running rake release instead, but it's my gem, not
yours. ????
Okay, now we've released a version with our fix, but master still has a
vulnerability: we need to port the fix. So let's go back to master:
$ git checkout master
And then cherry-pick our fix over:
$ git cherry-pick 168d5f756221
There will be a conflict. The diff is a little weird, such is life. Go ahead
and fix the conflict, then commit:
$ git commit
And you're done!
Other considerations
If we had a CHANGELOG, we'd need to udpate that as appropriate, including
creating new sections for our 1.0.1 release.
Sometimes it's easier to fix things on master first, then backport to the new
branch. I prefer to do it the way I showed here.
You should probably try to get as many people to know that you've fixed the
bug. Tweet, blog, rabble-rouse, and possibly post to the ruby-security-ann
mailing list, which
was created to help Rubyists know about security releases of their gems.
If your gem is really widely used, you may want to actually register a CVE.
You can find information on this process here. I made one here.
I'm going vim-gan. Basically, I don't want to consume any source code which doesn't come from vim, or products that contain non-vim source code.
While I won't, some people I've talked to will also consume code produced in Eclipse, since recent scientific studies show that Eclipse users can't actually feel pain. This is enough of an ethical gray area that I'm staying away from it, but I can see that perspective.
I just can't stay silent while the software industry brutalizes source code like this. Not cool, everyone.
Hey everyone! I made a video today about Ember.js and accessibility. I had always been repeating what I'd heard about screen readers: they can't work with JavaScript. Turns out that's not exactly true. The video is the best way to show you:
Here's a rough transcript, if you don't want to watch it:
Hey everyone. I recently was checking out Ember.js, and more specifically, Discourse. Discourse is supposed to be a next-generation discussion platform, and it's an Ember app backed by a Rails-based JSON API. I mentioned to Ember co-creator Yehuda Katz that I wished Ember had 'a better accessibility story'. He asked me to elaborate, and I mentioned how my friends on Twitter were saying that Discourse wouldn't able to be inclusive because JavaScript-heavy apps can't be used by screenreaders. So he said, “Hit ⌘-F5.”
Whoah.
Hitting that key combination activates VoiceOver, a feature of OS X I didn't know existed. Basically, VoiceOver is able to interact with applications and allow keyboard navigation for those who are blind or sight-impaired. Neat! But what about in-browser? Turns out VoiceOver handles JavaScript heavy pages just fine.
I tried it out on http://try.discourse.org/, and it works pretty well! Not perfect, but you can manage. Yehuda also pointed me at WAI-ARIA, which is a W3C initiative. I'll let the site explain:
WAI-ARIA, the Accessible Rich Internet Applications Suite, defines a way to make Web content and Web applications more accessible to people with disabilities. It especially helps with dynamic content and advanced user interface controls developed with Ajax, HTML, JavaScript, and related technologies.
Neat! Basically, you can add markup to your HTML, and it will help applications like VoiceOver allow people to use your site more easily, even if it uses a lot of Ajax. This is super cool, and for me, mitigates one of my main complaints against using SPAs. Of course, WAI-ARIA isn't really Ember's job; you'll have to make the HTML be output in the right way yourself. But it makes me really happy to know that people with disabilities don't need to be left out of the rich-client portion of the web.
Due to the recent situation with Rubygems.org, a lot of people noticed that they rely on Rubygems.org when deploying. A lot of people advocate “vendor everything”, and while that's one approach, I actually don't think it's necessary. I think a lot of people think they rely on Rubygems.org more than they actually do, or, at least, aren't deploying with Bundler correctly. So let's talk about it.
Deploying with Bundler
Bundler has a --deployment flag to help you deploy code. Here's the documentation. If you use Bundler, and you don't use --deployment when deploying, you are probably Doing It Wrong. Let's try a brand new app, and see what happens:
We have no configuration for bundler. Makes sense, we never bundled. If we try to use --deployment now, we get an error:
steve at thoth in ~/tmp/foo
[1] $ bundle install --deployment ✘
The --deployment flag requires a Gemfile.lock. Please make sure you have checked your Gemfile.lock into version control before deploying.
This is because --deployment checks the lock. So let's make one:
steve at thoth in ~/tmp/foo
[16] $ bundle ✘
Fetching gem metadata from https://rubygems.org/...........
Fetching gem metadata from https://rubygems.org/..
Using rake (10.0.3)
Using i18n (0.6.1)
Using multi_json (1.5.0)
Using activesupport (3.2.11)
Using builder (3.0.4)
Using activemodel (3.2.11)
Using erubis (2.7.0)
Using journey (1.0.4)
Using rack (1.4.4)
Using rack-cache (1.2)
Using rack-test (0.6.2)
Using hike (1.2.1)
Using tilt (1.3.3)
Using sprockets (2.2.2)
Using actionpack (3.2.11)
Using mime-types (1.19)
Using polyglot (0.3.3)
Using treetop (1.4.12)
Using mail (2.4.4)
Using actionmailer (3.2.11)
Using arel (3.0.2)
Using tzinfo (0.3.35)
Using activerecord (3.2.11)
Using activeresource (3.2.11)
Using bundler (1.3.0.pre.5)
Using coffee-script-source (1.4.0)
Using execjs (1.4.0)
Using coffee-script (2.2.0)
Installing rack-ssl (1.3.3)
Using json (1.7.6)
Using rdoc (3.12)
Using thor (0.17.0)
Using railties (3.2.11)
Using coffee-rails (3.2.2)
Using jquery-rails (2.2.0)
Using rails (3.2.11)
Using sass (3.2.5)
Using sass-rails (3.2.6)
Using sqlite3 (1.3.7)
Using uglifier (1.3.0)
Your bundle is complete! Use `bundle show [gemname]` to see where a bundled gem is installed.
This of course does hit the network, and does all the usual things. Your application normally has a lock already and has it checked in. We still don't have a config:
steve at thoth in ~/tmp/foo
$ cat .bundle/config
cat: .bundle/config: No such file or directory
So, this is the state our app would be in before deploying. At this point, it's just like we did a git pull or an scp to our server. So let's install for deployment:
steve at thoth in ~/tmp/foo
$ bundle install --deployment
Fetching gem metadata from https://rubygems.org/.........
Fetching gem metadata from https://rubygems.org/..
Installing rake (10.0.3)
Installing i18n (0.6.1)
Installing multi_json (1.5.0)
Installing activesupport (3.2.11)
Installing builder (3.0.4)
Installing activemodel (3.2.11)
Installing erubis (2.7.0)
Installing journey (1.0.4)
Installing rack (1.4.4)
Installing rack-cache (1.2)
Installing rack-test (0.6.2)
Installing hike (1.2.1)
Installing tilt (1.3.3)
Installing sprockets (2.2.2)
Installing actionpack (3.2.11)
Installing mime-types (1.19)
Installing polyglot (0.3.3)
Installing treetop (1.4.12)
Installing mail (2.4.4)
Installing actionmailer (3.2.11)
Installing arel (3.0.2)
Installing tzinfo (0.3.35)
Installing activerecord (3.2.11)
Installing activeresource (3.2.11)
Installing coffee-script-source (1.4.0)
Installing execjs (1.4.0)
Installing coffee-script (2.2.0)
Installing rack-ssl (1.3.3)
Installing json (1.7.6)
Installing rdoc (3.12)
Installing thor (0.17.0)
Installing railties (3.2.11)
Installing coffee-rails (3.2.2)
Installing jquery-rails (2.2.0)
Using bundler (1.3.0.pre.5)
Installing rails (3.2.11)
Installing sass (3.2.5)
Installing sass-rails (3.2.6)
Installing sqlite3 (1.3.7)
Installing uglifier (1.3.0)
Your bundle is complete! It was installed into ./vendor/bundle
Post-install message from rdoc:
Depending on your version of ruby, you may need to install ruby rdoc/ri data:
<= 1.8.6 : unsupported
= 1.8.7 : gem install rdoc-data; rdoc-data --install
= 1.9.1 : gem install rdoc-data; rdoc-data --install
>= 1.9.2 : nothing to do! Yay!
We still hit Rubygems.org, but note now that it said “Installing” rather than using. That's because we now have everything vendored:
$ cat .bundle/config
---
BUNDLE_FROZEN: '1'
BUNDLE_PATH: vendor/bundle
BUNDLE_DISABLE_SHARED_GEMS: '1'
steve at thoth in ~/tmp/foo
$ ls vendor/bundle/ruby/1.9.1
bin cache doc gems specifications
steve at thoth in ~/tmp/foo
$ ls vendor/bundle/ruby/1.9.1/gems
actionmailer-3.2.11 multi_json-1.5.0
actionpack-3.2.11 polyglot-0.3.3
activemodel-3.2.11 rack-1.4.4
activerecord-3.2.11 rack-cache-1.2
activeresource-3.2.11 rack-ssl-1.3.3
activesupport-3.2.11 rack-test-0.6.2
arel-3.0.2 rails-3.2.11
builder-3.0.4 railties-3.2.11
coffee-rails-3.2.2 rake-10.0.3
coffee-script-2.2.0 rdoc-3.12
coffee-script-source-1.4.0 sass-3.2.5
erubis-2.7.0 sass-rails-3.2.6
execjs-1.4.0 sprockets-2.2.2
hike-1.2.1 sqlite3-1.3.7
i18n-0.6.1 thor-0.17.0
journey-1.0.4 tilt-1.3.3
jquery-rails-2.2.0 treetop-1.4.12
json-1.7.6 tzinfo-0.3.35
mail-2.4.4 uglifier-1.3.0
mime-types-1.19
Okay! Neat. So we've Vendored Everything… what happens if we bundle again?
steve at thoth in ~/tmp/foo
$ bundle
Using rake (10.0.3)
Using i18n (0.6.1)
Using multi_json (1.5.0)
Using activesupport (3.2.11)
Using builder (3.0.4)
Using activemodel (3.2.11)
Using erubis (2.7.0)
Using journey (1.0.4)
Using rack (1.4.4)
Using rack-cache (1.2)
Using rack-test (0.6.2)
Using hike (1.2.1)
Using tilt (1.3.3)
Using sprockets (2.2.2)
Using actionpack (3.2.11)
Using mime-types (1.19)
Using polyglot (0.3.3)
Using treetop (1.4.12)
Using mail (2.4.4)
Using actionmailer (3.2.11)
Using arel (3.0.2)
Using tzinfo (0.3.35)
Using activerecord (3.2.11)
Using activeresource (3.2.11)
Using coffee-script-source (1.4.0)
Using execjs (1.4.0)
Using coffee-script (2.2.0)
Using rack-ssl (1.3.3)
Using json (1.7.6)
Using rdoc (3.12)
Using thor (0.17.0)
Using railties (3.2.11)
Using coffee-rails (3.2.2)
Using jquery-rails (2.2.0)
Using bundler (1.3.0.pre.5)
Using rails (3.2.11)
Using sass (3.2.5)
Using sass-rails (3.2.6)
Using sqlite3 (1.3.7)
Using uglifier (1.3.0)
Your bundle is complete! It was installed into ./vendor/bundle
The bundle command uses our config settings in .bundle/config, which re-runs with the vendored bundle.
Wait, I thought you said NO to Vendor Everything!!!
Well, here's the deal: you can Vendor Everything on your server, which means that we're not committing gems into source control, and then pushing that huge mess over the network.
Let's re-examine --deployment in the context of our two strategies: scp and git.
git
If you deploy with git, it's two ways: sshing into the server and running a git pull, or by doing a git push. In both cases, you'll have some sort of post-deploy hook that manages the rest of the deploy.
This scenario is exactly as shown above: just make sure that your first bundle on deploy is using --deployment, and you're Good To Go. Each next deploy won't hit the network. Rock on.
scp
If you use scp in some way to deploy, then you're getting a new copy of the application every time, so that bundle full of gems won't work. You need one more flag: --path
steve at thoth in ~/tmp/foo
$ rm -rf vendor
steve at thoth in ~/tmp/foo
$ bundle install --deployment --path ../vendor/bundle
Fetching gem metadata from https://rubygems.org/.........
Fetching gem metadata from https://rubygems.org/..
Installing rake (10.0.3)
Installing i18n (0.6.1)
Installing multi_json (1.5.0)
Installing activesupport (3.2.11)
Installing builder (3.0.4)
Installing activemodel (3.2.11)
Installing erubis (2.7.0)
Installing journey (1.0.4)
Installing rack (1.4.4)
Installing rack-cache (1.2)
Installing rack-test (0.6.2)
Installing hike (1.2.1)
Installing tilt (1.3.3)
Installing sprockets (2.2.2)
Installing actionpack (3.2.11)
Installing mime-types (1.19)
Installing polyglot (0.3.3)
Installing treetop (1.4.12)
Installing mail (2.4.4)
Installing actionmailer (3.2.11)
Installing arel (3.0.2)
Installing tzinfo (0.3.35)
Installing activerecord (3.2.11)
Installing activeresource (3.2.11)
Installing coffee-script-source (1.4.0)
Installing execjs (1.4.0)
Installing coffee-script (2.2.0)
Installing rack-ssl (1.3.3)
Installing json (1.7.6)
Installing rdoc (3.12)
Installing thor (0.17.0)
Installing railties (3.2.11)
Installing coffee-rails (3.2.2)
Installing jquery-rails (2.2.0)
Using bundler (1.3.0.pre.5)
Installing rails (3.2.11)
Installing sass (3.2.5)
Installing sass-rails (3.2.6)
Installing sqlite3 (1.3.7)
Installing uglifier (1.3.0)
Your bundle is complete! It was installed into /Users/steve/tmp/vendor/bundle
Post-install message from rdoc:
Depending on your version of ruby, you may need to install ruby rdoc/ri data:
<= 1.8.6 : unsupported
= 1.8.7 : gem install rdoc-data; rdoc-data --install
= 1.9.1 : gem install rdoc-data; rdoc-data --install
>= 1.9.2 : nothing to do! Yay!
steve at thoth in ~/tmp/foo
$ cat .bundle/config
---
BUNDLE_FROZEN: '1'
BUNDLE_PATH: ../vendor/bundle
BUNDLE_DISABLE_SHARED_GEMS: '1'
steve at thoth in ~/tmp/foo
$ ls vendor
ls: vendor: No such file or directory
steve at thoth in ~/tmp/foo
[1] $ ls ../vendor ✘
bundle
steve at thoth in ~/tmp/foo
$ bundle
Using rake (10.0.3)
Using i18n (0.6.1)
Using multi_json (1.5.0)
Using activesupport (3.2.11)
Using builder (3.0.4)
Using activemodel (3.2.11)
Using erubis (2.7.0)
Using journey (1.0.4)
Using rack (1.4.4)
Using rack-cache (1.2)
Using rack-test (0.6.2)
Using hike (1.2.1)
Using tilt (1.3.3)
Using sprockets (2.2.2)
Using actionpack (3.2.11)
Using mime-types (1.19)
Using polyglot (0.3.3)
Using treetop (1.4.12)
Using mail (2.4.4)
Using actionmailer (3.2.11)
Using arel (3.0.2)
Using tzinfo (0.3.35)
Using activerecord (3.2.11)
Using activeresource (3.2.11)
Using coffee-script-source (1.4.0)
Using execjs (1.4.0)
Using coffee-script (2.2.0)
Using rack-ssl (1.3.3)
Using json (1.7.6)
Using rdoc (3.12)
Using thor (0.17.0)
Using railties (3.2.11)
Using coffee-rails (3.2.2)
Using jquery-rails (2.2.0)
Using bundler (1.3.0.pre.5)
Using rails (3.2.11)
Using sass (3.2.5)
Using sass-rails (3.2.6)
Using sqlite3 (1.3.7)
Using uglifier (1.3.0)
Your bundle is complete! It was installed into /Users/steve/tmp/vendor/bundle
The --path flag controls where the bundle is located. In this case, we store it one directory up. Now, when we copy new versions of the code over, it will use the bundle location that stays the same, and all is peachy keen.
I am told by several people that this is what the Capistrano/bundler recipe does by default, so if you're using that, you're already doing this.
One Tiny Weakness
There is one small weakness of this approach compared to Vendor Everything: as an engineer, it's your tradeoff to make.
This way of using bundler will hit the network when you deploy for the first time after updating your bundle. The cache on the server has to update in this case, so it will go fetch the new gems. So here's the scenario:
I update my bundle. It gets the new versions of the gems.
Rubygems.org goes down.
I need to deploy a branch that has the new versions in it.
In this case, if you had Vendored Everything, the hit to Rubygems.org would have happened during step 1, and so things would work. If you used this strategy, it would have hit locally, so you could have developed, but then when deploying, it'd hit again to update the bundle on the server, and so it wouldn't.
In these situations, you can temporarily switch to Vendor Everything, since you have the bundle installed locally: just copy your local gems over to vendor/bundle and you're done. This may or may not be too much of a hassle. When I examine the downtime of Rubygems.org, I think it's worth it to not mess up my git repository with all that gem code. You might not. Do whatever you need to do, but now you know how to not rely on Rubygems.org for every deployment.
An addendum
I got an email mentioning one more issue with the 'vendor everything' strategy:
Basically, your compiled files get vendored, so if you're, say, developing on Mac and deploying on Linux, you might end up with compiled native extensions for the totally wrong architecture.
Rails' greatest strength is “Convention over Configuration.” Basically, Rails makes a ton of default choices for you about how things should be named, where they should go, and other things. This is why working with Rails is so productive for an experienced developer: just follow the Golden Path and everything comes together quickly. It's second greatest strength is its 'full stack' nature: Rails includes everything that you need to get going. It does not attempt to be 'minimal' in any way. Put these two things together, and a Rails app makes lots of assumptions about everything in your app.
But what happens when a significant number of people don't use those defaults?
First, let's talk about the actual default stack. Since Rails is omakase, I'm going to call this the 'Omakase Stack.' Roughly, this stack contains:
ERB for view templates
MySQL for databases
MiniTest for testing
Fat Models, Skinny Controllers
There has been a 'second default stack' brewing for a while now. I'm going to call this the “Prime stack”. This stack's approximate choices are:
Haml for view templates
PostgreSQL for databases
Rspec/Cucumber for testing
Skinny models, controllers, and a service layer
This is great! As David says:
That doesn't mean patrons can't express exceptions. Substitutions are allowed, within reason. So you don't like test/unit? No problem, bring your own rspec. Don't care for CoffeeScript as a dessert? Delete a single line from your Gemfile.
A considerable minority uses a stack like this. It's important that the Prime Stack isn't exact: you might not use Cucumber at all, for example, or maybe you don't have a service layer. The important part, though, is that this is the new default you build preferences off of: a big group of people assumes that you don't use MiniTest, you use Rspec, and the choice is Cukes or Capybara. This creates the 'secondary default' effect.
Dual Stack Ecology
So where's the beef? Well, before I get into it, I want to make one thing explicit: I am not taking a position on which stack, if any, is 'better.' I'm not interested, at least in this post, at placing them into any kind of order. That's for another time. What I want to talk about here is about the effect that a second default stack has on the overall Rails ecosystem.
Looking at history, the Prime Stack was born from the Omakase Stack. A certain group of people noticed that they were changing a significant amount of the stack most of the time, and so the Prime Stack came into existence. Since this is a historical progression, many blog posts got written describing the delta: take “Waiting for a Factory Girl”, for example.
Here at thoughtbot, we’ve had it with fixtures.
But wait! Then, years later, you get a discussion on how factories are bad. My own “Why I don't like factory_girl” is an example:
So that's what it really boils down to: the convenience of factories has set Rails testing strategies and software design back two years.
Maybe a bit strongly worded. Anyway, the people who have created the Prime Stack also happen to be those who are vocal, blog a lot, and speak at a lot of conferences. Everybody loves a polemic. If I submitted a talk to RailsConf called “Why Fixtures are :metal:” this year, people would assume that I was being sarcastic. People always assume that things are slowly getting better: If the original advice was “use fixtures” and the new advice is “use Factories” and the newest advice is “use neither”… there's no way that the “right answer” is “use fixtures,” right? Isn't that so 2007?
For those of us who've been around Rails for a long time, this discussion is good, and interesting. I was able to search for that Factory Girl post by memory, because I remember reading it when it was posted. I've written tons of Rails apps in the past 6ish years, and formed my own opinions on what tools I want to use due to that experience.
But what about someone starting Rails today?
Beginners lose out
Someone told me earlier today that “Starting Rails today is like starting to watch a soap opera in the 7th season.” Like most witty statements, there's both truth and falsehood here. But, as a newcomer, how do you pick between the Omakase Stack and the Prime Stack?
This is the crux of the matter for me. If today, you tweeted that you were building a Rails app with fixtures, people would tell you to Stop It. I saw it happen this morning. This is again, because a very vocal minority of Rails developers use the Prime Stack, and are really passionate about their choice over the Omakase Stack. That's great.
The problem is this: it puts a pretty heavy burden on a new developer: they have to learn the Omakase Stack and the delta. At the moment when someone is new, and barely knows anything, we throw them into making more choices. This is the moment when they're least likely to be able to properly evaluate which option they need to choose. It's not just “You're new to Ruby, now learn Rails,” it's now “You're new to Ruby, now learn Rails, and RSpec, and Cucumber, and use this advanced modeling technique that we agree Rails discourages. Oh, and your only help is all these blog posts. Have fun!”
As I said at the beginning of this post, Convention over Configuration is Rails' greatest strength. So when a new person starts, the first thing we do is tell them to do a bunch of configuration. Any reasonable developer could easily ask “If these options really are so superior, why do I have to include them? And which do I choose? I'm drowning!”
Helping People Build Better Apps
We all want to write better, more maintainable apps. And many people (including me) are trying to push the state of the art forward in this regard. But we're leaving the majority in the dust. Rather than trying to push the boundaries by choosing new techniques, maybe we should be helping people get better at existing ones, too. Because we need both approaches. There's already so much to learn when building a Rails application, adding more gems and techniques to the pile just isn't going to help.
Once those beginners become intermediate, then they can figure out how new techniques solve the particular pains that they feel while going with the default flow. But skipping straight to the tool that solves my pain without feeling that pain the first place doesn't help, it just confuses.
Let's keep pushing those boundaries, but let's not require someone to catch up on a few hundred episodes before they can understand what's going on.
http://whytheluckystiff.net/ has re-ignited a discussion about _why, and while I have tons and tons and tons of things to say on the subject, this is all I can give you for now: _why is a role, not a person.
For a long time, I was talking about a separation between _why and the person who played him. I still have not yet figured out how to refer to that person, and so for now, I've settled on Subject. So I'll be using that description for the rest of this post.
All the world's a stage
All the world's a stage,
And all the men and women merely players:
Shakespeare, “As You Like It,” Act II Scene VII
One of the things I've learned as I've gotten a little older (21 days till 27!) is that I'm a different person in different situations. This may seem painfully obvious, but let's expand on this a bit: when I'm on stage at a Ruby conference, I'm one person, when I'm by myself at home, I'm a different person, online, I'm a different person, and when I'm spending time with someone I care about, I'm someone else. Each of these contexts produces different behavior, mannerisms, and vocabulary. When I'm speaking with someone who's read French Postmodernism, I use very different terms than when I'm speaking with my Republican 'Murican relatives.
There's absolutely nothing wrong with this, but it really suggests that 'essentialism' is not a thing that can properly model the world. “Essentialism” is a position that states that there is some set of attributes that one can possess to 'be' something. So, for example, let's take a Zebra: some essential attributes may be:
So what makes a zebra a zebra then? I don't want to get into it, but basically, I take the position that you can't define something by what it is, you have to define it by what it does. I'm punting on why this is, again, I want to talk about _why, so please give me a charitable reading and take this as axiomatic for now.
If we're defined by what we do, then it makes sense that we can contextualize what we do: we can take on different roles at different times. When I program in Ruby, I am a Rubyist; when I ride a bike, I am a cyclist, when I have a child, I am a father, when I love and am loved, I am a lover.
When you treat programming as an art, when you treat software as an artistic medium, when you spread joy and whimsy with what you do, you are _why.
It just so happens that Subject was the person who created this particular role, as was the one who best played it. That doesn't mean that we can't also try to take that role on from time to time, try out out, see if it fits, wiggle our toes a bit.
_why vs. Subject
This is one of the reasons I think the concept of _why is important: Subject was trying to teach us something. They were creating an abstraction, or, maybe, an indirection. By keeping themselves private, they put an emphasis on the distance between _why-as-role and _why-as-person, since the second doesn't exist. The focus was, as it should be, on the role.
This is also why it's wrong to 'track _why down.' This doesn't make any sense. Subject is not _why, they just played them on TV. I also think that's why it's not wrong to keep talking about _why, as long as we make sure that we're actually talking about _why, and not about Subject. Subject deserves their privacy.
I also don't mean that I want someone to literally 'take up the mask' of _why and publish under that name. What I mean is that if we want to take on the lessons that Subject taught us through their persona, we should periodically have becoming-_why moments.
On Sloppiness and blog posts written in haste
@steveklabnik hand write it, take a picture of it. leave the mistakes in.
I work on a lot of different things. Sometimes I get asked how I find the time, but there's another aspect not related to time management: most of my projects build off of each other.
Draper and RequestStore
This issue got filed in Draper. What Draper does isn't important, but it did have this code in it:
def self.current
Thread.current[:current_view_context] ||= build_view_context
end
def self.current=(context)
Thread.current[:current_view_context] = context
end
Basically, storing stuff in a Thread local variable. This is great, normally, except when you run one of those new hot servers like Thin or Puma. They have a different concurrency model, and so while you'd normally expect this Thread local to be nil on each request, it will persist between requests.
I could have just fixed this bug in Draper, but I figured that it might be useful to someone else, so I whipped up a little gem to do it: request_store, and used the gem in Draper. I tweeted about it.
Turns out that one of my friends had exactly this problem, and wasn't sure how to solve it. Now a bunch of people at Heroku are using my gem, and it's helping them in their work.
I wouldn't have bothered building it if it didn't scratch my itch, but by making it easy for other people to use, I helped them scratch their itch too.
Rust for Rubyists and Designing Hypermedia APIs
I've been working on a new iteration of Designing Hypermedia APIs that is more linear and like a traditional book. The issue is that I'd built the entire platform for the project myself, and now spitting out ePUB and PDFs would be hard.
Enter Rust for Rubyists: I'd been learning Rust, and there are very few docs since it's a new programming language. I had also seen Zed Shaw present at RuPy recently, and he showed off Orkestrix, which let him generate a site and a book for music stuff.
So I decided to write up what I'd learned as a tutorial on programming in Rust, and built it using the Orkestrix gear. I also tried out DPD after seeing others use it. It's only been up for two days, but I really like the combo, and today I pushed my first update out to the people who bought a copy already.
This has given me the confidence that the tooling process is good enough to use for DHAs, so I'll be doing all of that soon.
Rails and Resque
I started working on Resque because we needed to have a few production implementations of the new ActiveQueue system going before Rails 4 was released. I was already working on Rails to help get 4 out the door, and Terence was obviously swamped by all the things he works on. So helping out with Resque would help out with Rails.
bring_back_snowman and configuration
When I don't know how to do something, I often look at a gem that does what I'm interested in doing, and copy how it does it. I had never built a gem that allowed you to configure it using the config.something pattern before, so I peeked at a few gems to learn how.
I had also been thinking a lot about the Rails 4 release, and what was new since Rails 3. I was teaching a new batch of people Rails with Jumpstart Lab, my employer, and was explaining why we have form_for generate a ✔. This gave me an idea: What if you could configure what Rails used in this case?
These are just the most recent examples I can think of. I'd really encourage you to try this sometime: if you want to learn something, make a new little project. Make it small. Make it focused. Make it help you out with something bigger.
