As usual, this year's JSConf did not fail to deliver. The parties, the location, the talks, the food, it was incredible. Chris Williams (and the rest of the JSConf family) really put together an incredible event.
By far, the most controversial talk was Brian Ford's "Is Node.js Better". I must admit, I walked in skeptical. Brian Ford seems like a bright and reasonable guy, but is clearly not a noder.
I'm not going to transcribe what he said. If you want that, read his slides and watch the video when it comes out. This is my reaction and response, and says more about me than it does about Brian.
This is based mostly on my meat-brain memory, and some notes that I scribbled. It's quite possible that I may get some of the particulars wrong. However, rather than wait for the video to come out, I'd like to write this now, since the errors in perception are themselves useful information. If you notice any such error, please let me know.
Of course, there are limits to how deep a speaker can go into a topic in a 30 minute talk, and those limits are much shallower than how deep I can go in this blog post. So, I'd like to frame this response not so much as me-vs-Brian, but rather as just another public part of a longer multi-format conversation. My goal is not to produce more controversy than is useful, but to perhaps produce enough to keep things interesting.
Brian started his talk with a very insightful exploration of the nature of controversy. He talked about our fast brain and slow brain, and guaranteed that reading Thinking Fast and Slow would change your life for the better, or he'd refund the cost of the book.
In discussing the title of the talk, he made the point that every value judgement implies a comparison, and invites controversy. If presented with a choice of apples and oranges, and you remark that "oranges are healthy", then the implication is that apples are not (or at least less) healthy. So, the question "Is Node.js Better" raises the question, "Better than what?"
He went on to stress the importance of the scientific method, especially in cases where groupthink can grab ahold of a popular intuition. A powerful example of this is the idea of brainstorming, which has come under criticism in recent studies.
Of course, another fascinating point worth mentioning (which was not covered in Brian's talk) is that the primary source of the criticism is a survey of the results of 22 different studies with different methodologies, done in different environments, on different subjects.
I'm wondering what the differences are between the 4 where brainstorming worked, and the 18 where it didn't. It's especially worth noting that 18 instances of brainstorming's failure vs 4 instances where it was successful, does not imply that brainstorming is "usually" less effective: only that it's usually less effective in the studies surveyed. If you check the weather in Wisconsin 18 times in the summer, and 4 times in the winter, you might conclude it's usually tropical.
This survey was reported several times in the Journal of Personality and Social Psychology. More recently, these journal articles were summarized in the New Yorker and other popular magazines. To me, this smacks of "turns out"-ism. It might be that the popular intuition about brainstorming is wrong. It might also be that it's just incomplete.
Brian discussed the concept that organizations tend to perpetuate the problems that they were designed to solve. To my mind, this implies that we ought not to try to solve our problems with organizations, but rather, to solve them with chaos and disruption instead, wherever possible. Since it is often not possible, and because "disruption" becomes just another buzzword meaning a specific sort of organization, we should make it our goal not to "solve problems", but rather to find the problems that are hidden by our current assumptions.
"Solving" a problem, then, becomes more an exploration than a "fix this thing" exercise. An exploratory expedition stops exploring when it returns with an answer. Similarly, organizations that are designed to solve a specific problem (as opposed to organizations that are designed to, say, throw parties or make buttons) should have their own destruction built into their core constitution.
This principle of minimum institution has guided the choices we've made in npm and in Node.js, technologically as well as socially. It has shaped the community in subtle ways that are difficult to recognize from the outside. It's a principle that is close to my heart, and it's why Node and I have gotten along so well.
"Programming is a behavioral science." Couldn't agree more. However, the transition from "behavioral science" to concurrency as a way to deal with scarcity of compute resources felt a bit forced to me. From that point of view, everything is behavioral science, because everything we do is done out of some human need (or else we wouldn't do it.)
I think it's more precise to say that software development is behavioral science, because every interface is a human interface, and humans are even more unpredictable than hard disk platters.
There were a few points where I believe that Brian simply was either incorrect, misinformed, or perpetuating and reacting to the very groupthink that he is ostensibly setting out to move past.
There were a few justifications for using Node that Brian listed:
These are all pretty interesting justifications, though really, the first two are rationalizations for the third. For a talk that started out by diving into the science of hedonics and decision theory (albeit a somewhat Gladwell-esque popularization of it), I was a bit disappointed to see that he didn't go into more depth on this point.
"Node.js is more fun." This reminds Brian of Ruby a few years ago. It's important to remember, in the context of hedonics especially, that "fun" is a highly subtle term worthy of study on its own.
The problem with "Justifications for doing X" is that we do things primarily because of our emotions. Even the action of "being rational" is a thing that we choose to do because of a positive feeling attached to the self-image as a "rational person who does smart things", and the assumption that we'll get some pleasantly winful reward that feels good. When we really want to do something irrational, we usually have no problem reconciling the conflict with our preferred self-image as a rational individual.
Emotions are not just part of motivation, they are motivation. If you want to know why a person does something, don't ask them to explain why they do it; try instead to figure out how they feel. It's a much more challenging question, but it leads to much more interesting information.
When we have a good feeling about something (or even, when we decide something for some reason, but don't have any particularly strong feeling about it), our brain begins inventing rationalizations right away. The first one to pattern-match against "X because Y" is likely accepted as the "reason" we're doing something, regardless of how much the truth of "Y" actually affects the likelihood of "X".
Try asking a compulsive gambler why he goes to the casino. He might tell you about his "strategy" for roulette. Of course, this is bullshit. Roulette is a deterministically losing game; the more you play, the more you lose. Same with craps, slots, and (unless you're the best at the table) poker. It's the feeling of winning that he's chasing, and the feeling of the chase. If he was truly motivated by the desire for money, there are obviously much better ways of getting it.
When a person shares their justifications for doing something, one of two things typically happens, in the "fast-brain":
If you have some vegan friends, and some friends on the paleo diet, and feel like watching some fireworks, ask them in each others company why they eat the way that they do. No matter how polite they try to be, each will subconsiously perceive the statements of the other as an attack, and feel the need to defend their position. If they are wise enough to retreat from needing to convince the other, they'll most likely at least make a big show about "agreeing to disagree". (Of course, if they've been on this diet for a long time, they may have enough practice at these sorts of situations to handle them quite gracefully.)
If X is not just "do this action", but rather "feel this way", and you do not already feel that way, and the justification is not enough to incite this feeling (as justifications almost never are), then we label it "hype", and it raises the bar that we now require to take the thing seriously next time.
Like so many cognitive shortcuts, this is a really good move much of the time. After all, people's justifications are usually self-delusions as often as well-informed and rational reasons for doing or thinking anything. It makes perfect sense to be extra skeptical when we are at the risk of being influenced by it. If we have to be extra skeptical over and over again, we start to pattern match "X is good" into the hype category. Suddenly, it's not just that veganism or paleo is not for me, it's that the diet is a mark of a foolish person.
The net result is that anyone saying anything positive is likely to be labelled a "fanboi", and their statement called "hype", no matter what they say. This is a dangerous feedback loop that leads technology communities to stagnation, bitterness, and chest-beating.
The mind is such a wonderful thing! Mistakes in the mistake detection lead to potentially valid statements being discounted because they are presented along with invalid justifications, or presented by a low-status speaker, or are in conflict with already-held beliefs, especially when those already-held beliefs are a part of our Tribal Story.
Even worse, you have situations where we see a few "X is good because Y" justifications, deem them false, and then subconsciously internalize the fact that we gain status in our tribe by applying the "hype" label and discounting it, resulting in spiraling down into the toilet. Politicians and marketers have made a science of getting us to elevate arbitrary ideas into this Tribal Story, blinding us to any disagreement.
Birds fly. Fish swim. Humans make mistakes. It is our Super Power. We all do this.
We almost never change our minds. We are influenced in ways we are incapable of detecting (and will deny!) We are motivated by the behavior of people who look like us superficially, and believe (at least temporarily) literally everything we hear.
There is virtually no limit to the ability of the human mind to find new and creative ways to get out of touch with reality. I don't want to give the impression that I (or anyone) could delineate all the many subtleties of human cognitive error. Every one is so rich and complicated. Even the pattern-matching of "X-because-Y" is a serious oversimplification.
In the ancestral environment of adaptation, disagreement was often fatal. We've gotten a bit better at intelligent disagreement, but we still try unconsciously to come to an all-or-nothing agreement within our own heads, annihilating any "bad" idea entirely, and shrowding any "good" idea in a halo.
Whenever tempted to call something "hype" (or, even, "anti-hype"), I try to remember to ask the following questions:
<person I respect>was saying it?
This is part of the reason why I try to criticise Node.js and npm as harshly and often and publicly as I can. I do think that they're tremendously useful tools... but how can you believe me if I tell you that they're perfect, when I clearly know better? And if you can't believe me when I talk about Node.js, then what good am I?
If you're not writing systems that are well served by nonblocking IO, or an event loop/observer pattern approach, or if you don't think it's fun to make programs this way, then you probably shouldn't use it.
But, that aversion is worth looking carefully at that, no matter what you decide. V8 is an extremely fast virtual machine, and these days, a lot of programs are IO-bound.
And even if it's not, we tend to focus on "the right tool for the job", at the exclusion of "an awesome tool for the job". The opposite of "right" is "wrong", but the opposite of "awesome" is "boring", which is sometimes worse, especially if you're trying to maximize creativity.
I'm not sure how much research Brian did when he went out looking for justifications for using Node.js, but I don't think he really got at what actually is driving so many people to check out this thing.
So far, I felt like the setup was nice, but the criticism itself was kind of lackluster. We've heard it before. Node is hyped. The single-language stack isn't all it's cracked up to be. Etc. I was eager for the meat.
No sooner had I thought this, than the next slide said:
Aha! There it is. That primal "under attack" feeling, the visceral tension in my lower abdomen, the warmth on my face.
When I was younger (and still today when I'm not at my best, I must admit) the temptation was to hit back, or dismiss the message. Fight or flight. I think Brian probably lost a bunch of the audience with this slide. It sure did get my attention.
These days, when I feel that reaction, I try to leverage the added awareness that comes with the adrenaline, to be on the lookout for whatever comes next, because it is full of valuable information, especially if it is highly disagreeable. Beware of the moments when rationality tries to slip away: those are the times you most need it.
As I am somewhat dogmatic about endeavoring to reject reality less, I was very excited to hear what we might be missing.
Sadly, the justification for the "ignores reality" claim was not as pointed as I would have liked. It amounted to:
For sure, garbage collection is tricky, and the interaction between
Node.js and the V8 garbage collector could probably be improved. But to
call it a lie is a bit silly. Garbage collection doesn't pretend that
you have infinite memory; that's virtual memory. Garbage collection
pretends that you don't have to
free memory in order for it to be
One of the biggest pain points of non-managed languages is having to explicitly free memory. The biggest pain point in managed languages is having to deal with a garbage collector. There may be some sort of approach to memory management that isn't collected or manual, but better than either. If so, I don't know of it, and it doesn't matter anyway, because that is a language design problem, and Node.js is not about designing a language. If you have a beef with garbage collection, take it up with V8 and TC-39.
I'd really like more details about what exactly Brian was referring to on this point about IO "always blocking somewhere" and in what way Node.js rejects that reality.
This was a valid concern in 2010. I said at the time that it would not be an issue in a few years. It's not an issue today.
Since the birth of Node.js, a lot of libraries have been built. There are 8500 modules on npm. There are bindings to every popular database, sophisticated test frameworks, template libraries, HTTP routers, and so on. It's easy and fun to write modules, and so people have written modules. This approach is remarkably valuable.
That being said, module discovery and visibility leaves a lot to be desired. npm has grown much faster than Rubyforge or PyPI, much earlier in the life of Node, and so we have hit these problems earlier.
Regarding tools, things are coming along quite nicely.
Post-mortem debugging with MDB just landed in the master branch, and will be in node 0.7.8. (Node v0.6 has had this for a while already.) Mad props to Dave Pacheco for that. Seriously, if you've never seen it in action, it's amazing. In all my years using PHP, I can't even count the number of times I'd desparately wanted this sort of thing. The zend IDE had some pretty good debugging capabilities, but Dap's jsstack stuff is truly magical. Call me a fanboi, I don't care: believe the hype. It's fantastic. The only sad point is that it's only available in SmartOS, but it's not exactly surprising that the OS made by Joyent has some special love for Node.js ;)
The DTrace support in Node is also impressive. Again, not surprising, given the fact that so many DTrace heavyweights work at Joyent, but the go-to DTrace library for node is not written by a Joyent employee. It's Chris Andrews' node-dtrace-provider module. Anarchy > curation, yet again.
The depth and breadth of information that can be provided by DTrace, and the great work by Brendan Gregg and others at Joyent to actually massage that data into a format that humans can easily consume, is absolutely phenomenal.
Similar work on run-time analytics and post-mortem debugging is underway at Microsoft, I'm told. Windows users: stay tuned.
The built-in node debugger client is more my style, but node-inspector hooks into the Chrome debug tools, which imo blow away most visual debuggers available. (They even have heap analysis tools!) There are also plugins for Eclipse to provide stepwise debugging and analysis, and a lot of existing Eclipse users are more at home there.
Bold claim. One that I was excited to see ample justification for. What are the last 15 years of Ruby mistakes? How can we avoid making those same mistakes? How did ruby fix them? What can we learn from ruby's experience?
Brian's main justification of this point was that the cluster module is a repeat of Phusion Passenger.
It's not entirely clear whether he was referring to TJ's "cluster" package, or the built-in "cluster" module in v0.6, or the much improved "cluster" module in node v0.7. None of them are a particularly close reproduction of passenger, though I suppose that TJ's is probably the closest. But it's not as if any of these are an Apache plugin, or tied to a specific web framework. The analogy is a stretch, to say the least.
What's even less clear is how Phusion Passenger, a program written in 2009, and widely recognized as the most effective and popular way to deploy the most effective and popular Ruby application framework, can possibly represent a "mistake", let alone 15 years of mistakes.
I can only conclude that I do not understand what he was trying to say, because otherwise, I cannot make sense of it. Maybe there was something else called "passenger" in the Ruby world once upon a time, which did kernel load balancing, IPC, and not much else. Perhaps this "passenger" was deemed a mistake, and is now gone.
Show me a ruby mistake we're reproducing, and I'll make sure it gets fixed.
Brian declared that process concurrency will never be able to scale adequately. I cannot accept this without data. Instead of data, he gave some hand-wavey assertions about garbage collection getting out of control.
I'll assume that he was running low on time, and perhaps had to cut out the histograms and demonstrations. So let's leave that question open.
Of course, spawning one process per request, or one process per IO or timer, is completely unreasonable. That's why CGI was not adequate, and no one runs a real web platform on bash. However, running a single process per core, and having multiple HTTP servers share the open socket, and then using an event loop for IO and timers, is remarkably effective.
If there is doom on the horizon, don't hint about it. Show me the evidence.
Finally! Something that genuinely sucks a lot in Node.js!
Domains is on the list for v0.8. I don't know that the first pass at domains will be the final and ultimate fix for this, but it'll be something to iterate on. This will make errors much more symmetric, and provide a much clearer chain of evidence.
The chain of evidence is also much improved by having post-mortem debugging of production systems. If you haven't yet, check out Bryan Cantrill's talk And It All Went Horribly Wrong.
So, this is important to me, it's important to Node.js users, it's important to the various companies paying the salaries of the node core team. It's reasonable to trust it's high priority.
If you want to know what's wrong with node, ask a node-core developer.
These are some of the things that are really problematic, some of them in a pretty deep way. Of course, these are the things I know about, so they're things that we're working on for future releases of Node.js. If there are things that you think belong on this list, let us know.
Some of them are very hard, and will require iteration. Some of them are pretty straightforward, and will be addressed soon. Some of them are not super difficult, but just lower priority, and won't get addressed for a while.
And of course, just the fact that Node.js is very young, and has some bugs. It'll no doubt get more, as we continue to work on it, but the goal is to change things in ways that make these bugs easier to spot and easier to fix.
It is not sufficient for Node.js to be better than any other platform. It must be so good that it permanently raises expectations in this space. It must continue to impress and delight users. There is so much work yet to do, it's mind boggling.
Laurels are not very comfortable to rest on.
If you've gotten this far, congratulations. I try to keep blog posts much shorter than this, but there was a lot to respond to here, and touched on several issues that are very close to my interests.
Brian's talk was fascinating. Even the mistakes were interesting. When the video comes out, I highly recommend watching it. Go get Thinking, Fast and Slow. I haven't read it, but I'm familiar with some of Kahneman's other work in behavioral economics, and I suspect it's every bit as fantastic as Brian claims.
I got a chance to talk to Brian after the talk, and so hopefully a lot of these points (at least, the specific technical details of what sucks in Node.js and what we're doing about it) doesn't come as a surprise to him. He seems like a genuinely nice and thoughtful person.
We're all humans. We all think wrong. We all love, and worry, and try to do interesting things. We all fail, and feel shitty, and try to explain why we're not a bad person for it. We all succeed, and want to brag about it. We tell ourselves stories, and sometimes those stories are true, but usually they're only just true enough to get by.
Try to remember this, the next time you're telling someone how awesome (or over-hyped) Node.js is, or any other thing you have feelings about, and they suggest that you may be a brainwashed idiot. They are just trying to protect something, as are you, and in the course of this protection, our instincts will cause us to fail.
Show that monkey brain who's boss! Slow down, breathe, and try to remember that this person who disagrees with you is not some big dumb idiot, but most likely a very thoughtful person who's trying to do their best to create truth and beauty in the world. If you don't learn what motivates them, how can you ever join forces?