Comments on: Civilization, B.S.O.D.

By: TheRadicalModerate

TheRadicalModerate — Thu, 14 Jan 2010 06:12:13 +0000

@Felix–

Actually, it’s the Space Shuttle that had the five identical computers that voted on each other’s answers–except when the primary flight system code decided to occasionally put an entire minor cycle’s worth of I/O in the wrong cycle. (Gotta love those IBM guys from Oswego–nothing like a scheduled I/O architecture that you decide to implement on an asynchronous, interrupt-driven serial bus. For you aficionados of space program software bugs, this is the problem that caused the last-minute scrub of the first Shuttle launch.)

And it was really only four computers that voted on each other’s output–the fifth ran the Backup Flight System, which was independently coded so that if there was a software bug that took out the four primaries simultaneously, the crew could switch over to the fifth computer and still fly the vehicle.

Ah, it brings back fond memories: those carefree days of programming in HAL/S, a language with all the bizarreness of APL and the gracelessness of Fortran, to say nothing of the fun of hand-compiling AP-101 hex patches. (NASA would only let us recompile the whole system every few months. I think that this was their version of source code control.)

By: Brian

Brian — Mon, 11 Jan 2010 04:05:47 +0000

Programming has “Design by contract”. http://en.wikipedia.org/wiki/Design_by_contract

This starts to get at the feedback loops you describe. These contracts are typically enabled during testing cycles to expose issues and turned off when running in production to save resources.

Another developer tool is automated unit tests.
http://en.wikipedia.org/wiki/Unit_testing

These tests are used to gather feedback during the development process. They are extremely useful for pushing different scenarios through software subsystems and components to find potential issues.

Agreed, building robust, runtime based feedback systems is where we need to get, but these two solid software engineering tools help for now.

By: Walter Sobchak

Walter Sobchak — Mon, 11 Jan 2010 03:23:22 +0000

Good take, but ESR is ahead of you and has the plan: Complex Adaptive Systems

By: C++ Developer

C++ Developer — Mon, 11 Jan 2010 03:04:44 +0000

I think restaurants and patrons are more apt than carpenters and woodpeckers: If you saw many restaurants’ kitchens, you’d never eat there. But you don’t demand too see them, or even think about it (often). You presume it’s passed some minimal inspection and is fine. It, you know, seems fine, doesn’t it?

Of course, with software such minimal inspections often fall apart as the inspector can’t just measure bacteria. (And I’ve seen some pretty horrendous software that’s passed layers of independent review.)

So where does that leave us? Perhaps pretty much where we are. Perhaps companies should publish the tests they’ve run (though that gives the competition a lot of free work). And how meaningful would that be?

To bring forward and generalize your “moral railroad” thoughts: we (people and software) pretty much all want the right thing already: for the trains not to crash, the restaurant patrons not to get sick, the software not to crash. Further, we can measure how much we “want” each by how much we’ve spent (training or testing for people or software), though with diminishing returns. So we agonize (hopefully) over whether we’ve done enough. And here we are.

By: M. Report

M. Report — Mon, 11 Jan 2010 02:29:55 +0000

I have told you once, I have told you twice, what I tell you three times is true:
Software _and_
hardware redundancy;
Different designs for each of the six bits.

EMP is a greater
threat, and harder
to protect against;
Good business opp-
ortunity there.

By: ErikZ

ErikZ — Sun, 10 Jan 2010 22:19:28 +0000

“I say your civilization because as soon as we started thinking for you it really became our civilization…”
~Agent Smith

By: Felix Kasza

Felix Kasza — Sun, 10 Jan 2010 22:04:51 +0000

So assume the renderer gets feedback that there already is something where it wishes to paint the next bit of text. All that means is that the renderer will assume that the page creator _wants_ the overprinting effect — unless you have three separate renderers, written by three separate teams, with Chinese walls in between, and sharing not a single line of code (and running on separate OSes, too). In that case, you can let them vote, by majority, which colour a pixel should be.

We do things like that (in space vehicles, for example) — but even there, we do not do it completely. Apollo modules, for instance, had five computers, but they were the same model running the same code. _And_ the software they ran was as simple and minimal as possible. For something as complex and incomplete as the HTML spec, you can forget voting.

Felix.

By: Dave

Dave — Sun, 10 Jan 2010 21:30:47 +0000

You need to look into the Erlang programming language, developed by Erickson for programming telephone switches. Software components developed in Erlang are designed to crash cleanly in the presence of errors and be restarted automatically and quickly by other software components (called supervisors). That way you don’t have to try to detect and respond rationally to every possible failure case, but instead “just fail” and count on another piece of software to clean up and restart whatever portions of the system failed, transparently. Everything in an Erlang system has the sort of multi-layered feedback you are are talking about. It does take a fair amount of CPU cycles, and a somewhat unusual programming style, but the results are impressive. Using this sort of error-resiliant programming, Erickson achieved uptime percentages of up to “eight nines” (99.999999% availability, or a maximum of 1/3 of a second of downtime per year).

By: Micha Elyi

Micha Elyi — Sun, 10 Jan 2010 21:22:28 +0000

Hey, TheRadicalModerate, it's not just any old chestnut, that's a chestnut of Gerald Weinberg's invention published in Computer information systems: an introduction to data processing by Gerald M. Weinberg and Dennis P. Geller, Little, Brown computer systems series, 1985 p.113.

By: JMHawkins

JMHawkins — Sun, 10 Jan 2010 20:47:44 +0000

As someone who writes software (probably even the software that renders the page you’re viewing) I can say that the “problem” with writing robust software is that it is a couple of orders of magnitude more expensive than writing software that works “most of the time.” It’s all well and good to say “well, than that’s how we should do it” but unless you’re paying for it, that doesn’t really help. If I decided to create a browser, or a phone, or the software for any other typical commercial product, the way suggested here, I’d go broke because nobody would buy it. Everyone would buy my competitor’s product that had 10x the features at 1/10th the price and crashed every couple of weeks.

The problem (and hence the solution) is with the folks writing the checks, not the folks writing the code. The comment about woodpeckers wiping out houses built like software is pretty funny, but the homeowner is the guy who wouldn’t pay more than $49.00 for the entire house (and there are a whole bunch of wannabe homeowners complaining that even $49 is too expensive – houses ought to be built for free, as a hobby I guess).

After a couple of generations of this, the bulk of software engineers don’t have the mindset to create really robust software (why should they? Nobody rewards it). Plus, the development of techniques to ensure robustness have lagged behind where they could be.

When your wife bought her iPhone, did she do it because iPhones have a reputation as being more reliable than a standard old dumb cell phone that maybe can do texting? Or did she buy it because it has that touch screen where you can slide stuff around with your finger and download all kinds of cool and useful apps?

It has to be a consumer revolution to fix this problem, but I really don’t see the consumers revolting any time soon.