QCon San Francisco 2009 – Part III of III

-=-==-=-=-=-=-=-=-=-=-=-=–=-===—=–===–=–=-=-=-=–=-=
Day 3
=-===-=-=-=-=-=-==-=-=-=-=—=-=-=-=-=-=-==-=-=-=-=-=-=-=

Did you every create those fancy borders using ascii dashes and equals? That’s one of my favorite things to do in a text editor.  I hold my index finger and middle finger over the pair and randomly punch each one until I get about 80 of them… every border comes out unique this way. It’s awesome, you should try it. Really. I’ll wait. Pretry cool, huh? Well, the keynote by Oracle’s Dave Chappel wasn’t nearly as exciting as creating dash-equal borders. These are some of the tweet’s I issued:

It might be hard to believe, but the Oracle keynote is not good on my laptop battery as I’m surfing wildly. #qcon

That was true, I burned through 25% of my battery. If you think that was clever, you should have seen my next tweet. Golly, this was the most clever one of them all!

Oracle’s cloud strategy is more confusing than writing “Hello, World” in malbolge. #qcon

How many of  you knew what malbolge is? Seriously? Show of hands? I don’t believe you. Well, if you did then I am impressed. Consider this malbolge program:

(‘&%:9]!~}|z2Vxwv-,POqponl$Hjig%eB@@>}=<M:9wv6WsU2T|nm-,jcL(I&%$#”
`CB]V?Tx<uVtT`Rpo3NlF.Jh++FdbCBA@?]!~|4XzyTT43Qsqq(Lnmkj”Fhg${z@>

What that program does is print “Hello World!” to the console, with no newline I might add. I mean, it’s pretty clear what it does… right? Google malbolge, I guarantee it will wast 10 minutes of your time as you digest that material. Freaky. Was I talking about the Oracle cloud strategy? I tweeted how confused I was and someone actually sent me a link (via Twitter) explaining Oracle’s strategy. Thanks Rex! Here it is:Oracle Keynote Cloud Expo 11-04-09

View more presentations from Rex Wang.

The presentation is seriously need of de-corporatizing as there is way too much going on. IMHO, its rubbish. In a nutshell, Oracle is offering options… private or public clouds, your choice. They will sell you the infrastructure either way.  Well, actually,  they partner with Amazon’s EC2 service for the public offering, which will run Oracle’s software stack underneath. To me—and I must admit I have no actual programming/implementation experience with a true cloud infrastructure—this is a less clear message than Microsoft provides with Azure. Azure sounds so much sexier. Being in the cloud with Azure sounds sexy. Being in the cloud with Oracle sounds kind of boring. Why were there no sessions on Azure at #qcon? That sucks.

Did you see the cool em-dashes? On Windows, you can create em-dashes a few ways. The first way is to hold down the alt key and type 0151. That’s crap. Most “Microsoft” applications will allow you to hit Ctrl-Alt-Numpad hyphen, which is more intuitive. On the Mac the finger contortion is shift+option+dash (next to the zero, up top). Yep, you got it. That works in most Mac software. Try it!

I’m in the long process of reading Gödel, Escher, Bach by Douglas Hofstadter. My friend John lent this to me and I must admit it’s a goddamn chore to get through. But, every now and then it makes me think, hard. And I get smarter! In this book, Dougy often challenges the reader to “Try it” to make sure the concepts gel. So, seriously, try the em-dash thing. Do you know when to use an em-dash? Do I?

Wait, did I get distracted again?

-=-==-=-=-=-=-=-=-=-=-=-=–=-===—=–===–=–=-=-=-=–=-=
Eric Evans
=-===-=-=-=-=-=-==-=-=-=-=—=-=-=-=-=-=-==-=-=-=-=-=-=-=

IMHO, Eric was best speaker at #qcon. I spoke to him afterwards and he said that this talk was new an mostly unrehearsed. In contrast to his first talk, which he’s done dozens of times. Nevertheless, it sounded good to me. Eric has been to my office a few times. Always a pleasure to hang out with him.

Eric challenges: modeling is not an up-front investment that pays off. It helps you get there in the first place. He also challenges the old agile adage: “The simplest thing that could possible work” is often interpreted as “quickest thing that could possibly work.” Eric says that it would be better stated as, “what is the most concise, clear, and most easily understood way to do this”… His point was to do that requires real work. It’s so sad that so many projects have failed because people did simple—and stupid—things. Oh, and don’t let me make you believe I’m not a victim of this from time to time.

Another notable quote: “Typical UML is not a good representation of the model as it is a visual representation of the program. A model, in contrast, is a collection of assumptions, rules, and choices that led you to write the program that way.” Well said, Eric.

Eric asks, how then do you reconcile Agile and Domain-Driven Design? Tech can distract us. Feature-orientation can fragment our model. Upfront analysis locks in ignorance. (If you’re thinking things are seeming a bit “disjointed”, then realize that I’m reformatting my notes… and sometimes I lose context!)

Consider this statement:

Need tools that let
us express models
without getting bogged
down in technical detail

vs.

Need tools that let
less skilled developers to use
parts made by smarter people

What Eric is saying here is really a criticism on elaborate libraries and frameworks that get created to dumb down the intricacies of a system so that “less skilled” developers can be productive. The problem is that visual programming and “compiled UML” are simply bad concepts. It seems pretty obvious. If you abstract away all the details, the “less skilled” developers will never get a real sense of  the system. They will work at an abtract level and ultimately fail to achieve mastery. Like, ever. To be truthful, language is our fundamental abstraction tool. What you talk about the domain reflects the model. Here is where DDD and Agile start to diverge. In agile, you get the right answer and that is enough (green bar!). In DDD, we want to get to the right answer for the right reason. Precious. Eric then said, to many laughs:

The way I work things is not like a waterfall, but more like a whirlpool… if we insist on violent water metaphors. #qcon

Eric then presented a specific process he and his people at Domain Language use with their clients to incorporate exploratory modeling and design into an Agile process. He called it “Whirlpool” and said that it wasn’t quite ready for prime-time, but I’ll keep a lookout and write more about it later. I don’t have the whirlpool graphic he showed, but I’ll try and get a hold of it.

Some more EE gems:

  • You are not done until you have three bad ideas.
  • If your model runs into one special case after another. Test the model with a new scenario.
  • We will always get our predictions of the future wrong,  but this does not make predicting the future a senseless endeavor. We just have to learn not to take our predictions too seriously. #qcon

Great presentation.

-=-==-=-=-=-=-=-=-=-=-=-=–=-===—=–===–=–=-=-=-=–=-=
M – Dynamic Language Grammar – Amanda Laucher and Don Box
-=-==-=-=-=-=-=-=-=-=-=-=–=-===—=–===–=–=-=-=-=–=-=

M is a language unto itself, it’s not the Lex and Yacc model where you have a spec and you spit out C on the other end.  You can build a SQL database with M. You can use M to generate ORM layers. You can dynamically validate data in your own programs. The cool thing about M is that it’s dynamic. It’s also a CLR language so it inter-operates with all of your code (assuming you target the CLR!). It allows you to define DSL’s and validate them at runtime, nay, traverse them, violate them, and do whatever you like. All in all, seems very powerful and the audience seemed to “get it” toward the end.

-=-==-=-=-=-=-=-=-=-=-=-=–=-===—=–===–=–=-=-=-=–=-=
Skeptical View of Language Workbenches – Glenn Vanderburg
-=-==-=-=-=-=-=-=-=-=-=-=–=-===—=–===–=–=-=-=-=–=-=

This talk was a bit of a rant. Glenn fully admitted this from the start and acknowledged this to his audience, who obviously were developers of language workbenches (I think the developer of Intentional Software was in the audience). Here are more or less my raw notes:

  • AspectJ-Bring meta-object concepts, instead of building it into the language like CLOS, you bolt it onto the language.
  • CLOS, deep meta-programming within the language was a bad idea?
  • DSL’s, internal ones, in Ruby community, have been flowering. Ruby has simple syntax… so it’s generally pretty easy to create a SAL directly within the Ruby grammar

Compared to AspectJ, which is an Arcane, complicated sub-language… only a few people understood it well enough to use effectively. Language Workbenches, overestimating programmers. You need to deal with a problem sideways using language workbenches. This is my favorite quote from the con!

There is a reason why Jedi Knights build their own lightsabers. Programmers should not use a tool that seems like magic to them. @glv #qcon

How awesome is THAT! Jedi knight references!

Eric Evans in one of his talks says, “language is our fundamental abstraction tool,” and EE seems to support Glenn’s skeptical outlook.

The counter-argument for Language Workbenches is described very well by Martin Fowler in his ’05 treatise on the topic, Language Workbenches: The Killer-App for Domain Specific Languages? In this essay, Martin describes language workbenches as systems that allow you to modify the compiled form of your code, not the text, not the machine code, but the model in-between. While I’m not a fan of visual languages, compiled UML, or the 4GL insanity of the 90’s, there is something interesting here and something future-predicting about how programmers will be programming in 20 years.

My personal prediction will see the rise of functional languages and machine intelligence (not self-awareness, yet!) and the job of the programmer will be to input “requirements” or logic rules via natural language, and the computer will generate all possible programs that can solve the problem presented. The programmer will then cull out and filter the solution that exactly fits and move on to the next problem. But this is just me talking. What do I know?

-=-==-=-=-=-=-=-=-=-=-=-=–=-===—=–===–=–=-=-=-=–=-===-=-=-=-=-=-=-=-=-=-=
Amazon S3: Architecting for Resiliency in the Face of Failures – Jason McHugh
-=-==-=-=-=-=-=-=-=-=-=-=–=-===—=–===–=–=-=-=-=–=-==-=-=–=-=-===-=-=-==
This talk was about Amazon’s Simple Storage Service and how it’s resilient to failures, like large-scale failures… like datacenter dying failures.

I don’t have too much to say about this talk. It was the last session of the last day and I was getting a bit tired. Kind of like I’m tired after writing three posts in my qcon experience!

Anyway, Amazon S3 has a goal of 99.9% uptime. I was surprised to hear this since I thought enterprise-class uptime wass 5 9’s (99.999%). Anyway, Jason went over the main reasons for failure and exhaustively went through the resolutions for each:

  • Human error
  • Acts of Nature
  • Entropy
  • Beyond Scale

I didn’t write down all of the resolutions, butyou can download the slides from the qcon website.

There was a lot of talk about Amazon’s gossip protocol. Amazon S3 uses a gossip protocol to quickly spread server state information throughout the system. This allows Amazon S3 to quickly route around failed or unreachable servers, among other things. Last year, gossip got out of control due to an error… bringing down the service infrastructure. Check out the statement. You can also check out Amazon’s service dashboard, and even subscribe to an RSS feed for it.

I did some quick Googling and couldn’t find much about the protocol, but I did read an interesting passage for a thesis titled, On the Correctness of Gossip-Based Membership Protocols:

The importance of scalability and fault-tolerance in modern distributed systems has led to considerable research in multi-cast gossip protocols. In a gossip protocol, each node forwards messages to a small set of “gossip partners” chosen at random from the entire group membership; traditional strong reliability guarantees are traded for probabilistic guaranties, potentially yielding greater scalability and fault tolerance. Nodes only stores a small random subset of the membership as maintaining complete membership views at each node is expensive. These protocols are subtle, and while they have been the subject of much simulation and analysis, formal proofs of key properties – in particular the probability of network partitioning – have remained elusive.

Jason admitted that the Gossip protocol doesn’t always work at scale and offered a link to the ElasticVapor blog which had some interesting things to say, including this statement:

One such example provided by wikipedia is in a network with 25,000 machines, it’s usage can find the best match after about 30 rounds of gossip: 15 to spread the search string and 15 more to discover the best match. A gossip exchange could occur as often as once every tenth of a second without imposing undue load, hence this form of network search could search a big data center in about 3 seconds.

Jason also recommended that once you have all of your fault tolerance scenarios implemented, you should turn off a datacenter—literally pull the plug—once a quarter. Make sure you don’t do this before you’re ready, however, as there may be some downtime ;-)

In conclusion…

Hope you weren’t waiting for more analysis on these topics. Hey, I ‘m just jotting down my notes. You can watch videos of a handful of these presentations and get the slides to most of them a the qconsf09 website.

I do hope you had a good time and congratulations for getting to this point! Looking forward to qcon next year and hope to see you there!