denise: Image: Me, facing away from camera, on top of the Castel Sant'Angelo in Rome (Default)
Denise ([staff profile] denise) wrote2011-03-29 05:34 pm
Entry tags:

Technical debt and the making of payments on it

I saw an entry posted the other day where someone said sie was disappointed with (among other things) DW's development pace slowing down: new features being released more slowly, things that we were working on delayed/postponed, etc. And there were totally some valid criticisms in there, don't get me wrong! (In fact, I'm not linking it because I don't want there to be an overwhelming impassioned defense of DW in hir comments.) But that's one criticism that made me realize I've been doing a poor job of explaining precisely what's been going on in DW development and why there's been a paucity of user-facing changes, which can look to an outsider like there's a massive slowdown going on in DW development.

The answer is at once both very simple and very complicated: we've spent the past six months or so concentrating on paying down our technical debt.

Technical debt, as detailed in that Wikipedia article, is the collective IOU you-the-developer write to your future self. Let me just do this fast now, and do it right later, you think: let me do the quick but messy way now and do the correct way later. But of course, the minute you ship, you're moving on to the next big thing, and you never do go back to fill in those FIXMEs and TODOs in the code, until the next time you have to touch that area and what would otherwise be a five-hour fix has turned into a twenty-hour fix because you have to "pay back" the debt you incurred last time.

More than that, you've got to pay the debt plus interest: it's a common truism that code you yourself wrote six months ago is as impenetrable as code written by a complete stranger, and you have to spend a great deal of time puzzling out what the heck you were thinking back then. (Code that is brilliant, flawless, and crystal clear when you write it slowly morphs into idiotic, bug-ridden, and clear as mud over time. This is a well-known process. I suspect pixies in the source code repository, working their anti-magic while nobody's looking.)

Now, like real-world, financial debt, sometimes there is prudent technical debt: sometimes it makes sense to incur those future obligations in order to accomplish something you otherwise couldn't have. We saw a lot of instances of that during the ramp-up to open beta, where we did it one way then to "get it out the door" and opened bugs for later refactoring so we didn't lose track of the debt we were incurring. (There's a good article about the technical debt quadrant that discusses these various types of technical debt, and when they might be useful.)

There's also scenarios where you don't realize you're committing to technical debt until later: you do something one way based on your best understanding at the time, and down the road -- due to new technology becoming available, new people joining your team with new strengths that allow you to work on things you couldn't before, new sources of funding appearing, new progress in hardware capabilities, etc -- you realize, oh, hey, we could have done it this way instead, and it would have worked much better.

In short, there's no way to completely eliminate technical debt in software design, and more than that, you wouldn't want to even if you could. Technical debt, like the responsible use of revolving lines of credit or obtaining a mortgage to buy a house in the real world, is an important part of the software lifecycle, and responsible use of technical debt is a tool that can enable a software project to succeed where they otherwise might have failed.

In a 'typical' two-year-old software project, the amount of technical debt you've built up -- if responsibly managed, which I like to think we've been pretty good at with the things we've made so far -- is fairly negligible, and you can probably get away with spending 10-20% of your coding time and effort to making payments on your technical debt without problems, thus leaving you 80-90% of your time to advance new user-facing features and improvements.

The problem is, Dreamwidth isn't a two-year-old software project. We've only been open for two years, true -- but when we opened, we forked from LiveJournal, which sprang into life in Brad Fitzpatrick's dorm room in 1999. In forking our code from LiveJournal's, we inherited that decade's worth of features, fixes, and improvements, but we also inherited a decade's worth of deferred technical debt. Dreamwidth isn't a two-year-old software project; it's a twelve-year-old software project. And the face of technology has changed quite a bit in those 12 years.

In the decade LJ was under development before we forked our code from theirs, there were of course payments being made on that technical debt; it's necessary in order to move forward. Really big payments, though -- the technical equivalent of paying down your credit card in one lump sum because you've come into a windfall -- were mostly deferred.

So, one of the things we need to do in order to move forward in a lot of instances, to take advantage of the advances that have been made in technology since various features and bits of site design were first coded, is to do all the work necessary to get to a place where we can take advantage of them.

I'll give you an example: when we forked from LJ, the code "out of the box" would only work on the Apache 1.3 series of web server software -- and early releases of 1.3, at that. (The web server is the program that runs on your machines and handles how to serve web pages to your browser when you ask for them -- if you don't have a web server running, nothing else works.)

The first release of Apache 1.3 was June 6, 1998 -- it was cutting-edge when Brad started LJ in 1999. The latest version that the LJ code would work with was (I believe -- I may be wrong) 1.3.17, released January of 2001. Subsequent versions of Apache wouldn't work with the LJ code -- it would cause horrible errors and prevent the site from running at all. To upgrade to later versions of Apache (which had more features, fixed security holes, and in general were more technically advanced) would have taken a lot of work on the code.

Even at that time, Apache was working on Apache 2.0 -- a much more technologically advanced version. The first 2.0 release was in March of 2000. The problem was, it was mostly incompatable with the optimization tricks used under Apache 1.3, and LJ code was highly optimized to take advantage of the 1.3 series. To port LJ over to the Apache 2 series would take an incredible amount of effort -- aka, technical debt.

By the time we forked the code, in mid-2008, Apache 1.3 was nearing the "end of life" -- the point past which Apache would refuse to support it, refuse to issue any additional updates, and generally say, look, c'mon, it's been nearly a decade, upgrade already. (It was formally EOL'd in February of 2010, but everyone agreed it was well past time.) So if we accepted the requirement for Apache 1.3, we were already tying our hands, and making it incredibly hard for our developers: the Apache 2 series had been the standard since at least the middle of the decade, and if someone wanted to run DW, either as a production website or to do development work, they would first have to spend hours actually downgrading their server in order to make the code work.

Before you guys even saw DW, before we could even get to the point where people could install and start hacking on the code, [staff profile] mark had to spend months making the code work under the Apache 2.0 series -- he had to sit down and pay the technical debt. The same thing happened with Perl, the language DW is written in. LJ had been written in an earlier version of Perl, and later versions had some backwards-incompatabilities; DW had to modernize before we could upgrade.

The things we need(ed) to do aren't all that obvious, of course. Those of you who came from LJ know that each page you see on LJ has the suffix ".bml" on the end of it. BML stands for "Better Markup Language" (or "Brad's Markup Language"), a templating system that produces the framework to generate a page. (Things like the site skins --Tropospherical, Celerity, Gradation, etc -- are a function of BML; the contents of the page are the same no matter what, but the templating system builds the various 'looks' of the page so you can just swap them in and out and the contents don't change but the display does.)

In 1997-1998, when Brad was working on FreeVote (his project before LJ) and the first iterations of what would become LiveJournal, there wasn't anything better out there, so he had to "roll his own". He got it to a place where it would work for LJ, and then -- because there wasn't any real need to advance things further, since it already did everything he needed it to do -- mostly stopped work on it. The face of the web changed a lot since that point (this may be a wee bit of understatement), and today there are a lot of incredibly powerful templating systems out there that do way, way more than BML does -- and are under active development, so there will be future awesomeness coming out of them.

Additionally, making people learn BML -- which is a fairly impenetrable system -- in order to contribute to DW would be silly -- it's a barely-documented custom language that only exists on less than half a dozen websites in the world. The other templating systems out there are in wide use, and it's way more likely that someone will already have the skill set necessary to contribute. (Not to mention, it'd be nice to have a templating system that has actual books written about it rather than a few web pages here and there.) Switching to a more standard version makes an incredible amount of sense.

It's also an incredible amount of work. (We've been plugging away at it, bit by bit, for over a year. At this point, the pages you see on DW itself are half generated by BML, half by Template Toolkit (the templating system we chose) -- you never see the difference, if we do our work right, but our end goal is to get rid of BML entirely.)

There are a lot of examples of things like this, from the "big project" issues to smaller things (like the need to take duplicated code -- where the same block of code is repeated in multiple places -- and move it into a function that can be called from anywhere instead, so that people only have to update one area instead of many). There are cases where functions and features that were revolutionary when they were implemented on LJ have aged over time while the technology has advanced in leaps and bounds -- a good example there is the implementation of the inline cut-tag expander; when "lj cuts" were first introduced on LJ waybackwhen, that technology didn't exist, or would've been too much of a pain in the neck to implement. Over time, it became easy; from becoming easy, it became expected, until the point where a site that didn't implemented it started to look clunky or backwards.

People have come to expect a lot of things from their social websites, in terms of 'standard' technical abilities -- I'm sure you can think of about a dozen things that other, more recently-designed websites implement that DW doesn't. We'd love to have those features. (I probably curse their lack about twice a week.) The problem, again, is that the backend code for those pages was written so long ago that before we can drop in those features and functions, we have to modernize everything. You'll never see the work. But we have to do it before we can move forward.

Some people have asked us why we committed to the LJ platform when we knew we'd be accepting the IOUs of ten years of the programmers who came before us. The answer is twofold:

1) In many cases, we were those programmers. We knew the code, knew what it could and couldn't do, had a reasonable perspective on what we were in for, and were incredibly familiar with the way it worked, the way it ran, and the way it was put together. (I say 'we', but I mostly mean [staff profile] mark there. I didn't get much into development until we started DW. But I still followed along with the technical discussions, and I had a pretty strong grasp of the technical end of things even though I hadn't been doing the coding myself.)

2) In addition to the ten years of technical debt, we also inherited the benefit of ten years of bugfixes, security fixes, architecture/performance improvements (in another 10 years when historians write the history of the early 2000s on the internet, I fully believe they will point to LJ as the technical pioneer that made a very great deal of Web 2.0 happen; the problems LJ solved back then are universal to any high-load system, and the solutions they/we came up with are still in use today), and feature development.

We believed, and continue to believe, that the LiveJournal system and code contains some of the most incredible social features out there, to the point where even today, ten years later, there is no other site that does everything the LJ code does and does it as well. A lot of that amazingness is buried, now, under a lot of "usability problems" that are actually relics of the fact that nobody went back to modernize things once the first draft was released. (To be fair, there are tons of usability problems that are actual usability problems, and were at the time the feature was released, as well. And I don't want to intimate that LJ-now is ignoring these problems either; they've been doing a lot of work on their own technical debt lately, as evidenced by the number of people who accuse them of not working on any new features either.)

One of our major goals with DW is to take the awesomeness that is inherent in the LJ codebase and bring it into the "modern era" of web design and function. We've made some great strides, but we're still only part of the way, and every time we set out to do something new, another whole chunk of problems that we have to address first pop up. It's the technical equivalent of having to learn to crawl before you can learn how to walk: before we can complete the new update page redesign, for instance, we need to do an incredible amount of work in order to make it possible to intermingle JQuery (the Javascript library that we need for the modernized widgets on the update page) with the existing Javascript the site uses. (Among many other things.)

So, when you see a code tour that's full of nothing but backend improvements, don't think of it as "DW isn't doing any feature development". Think of it as "DW is doing the necessary background work to enable awesome feature development in the future". The work we're doing now is going to pay off in the future, and it's going to allow us to do epic things.
sharpest_asp: Lady Vader (Leia) with saber (Default)

[personal profile] sharpest_asp 2011-03-29 11:50 pm (UTC)(link)
Thank you. I have been quite content...but I'm no developer. I am comfortable here, and if you say the back end needs cleaned up, great. But in taking the time to explain, so that even a non-tech like me can get it? Thank you.

[personal profile] feathertail 2011-03-31 07:53 pm (UTC)(link)
You did a good job with your explanation.>.>b Goddess knows I've got enough of that debt to pay off on my WordPress site.

I'm still concerned that DW is cliquish and impenetrable, socially and technically. The invite codes seem almost redundant, because the only way to ever get settled here is if you've got friends on the inside already (I came here blind and encouraged my friends to follow, but still). And OpenID, instead of being a gateway to let members of other sites participate, is a barrier that only the technically adept can pass, and it makes them into second-class citizens once they're here.

As it stands, Dreamwidth is great for safe spaces -- those may be its killer feature. But there aren't very many who can take advantage of it.
changeling: (Default)

[personal profile] changeling 2011-04-11 06:32 am (UTC)(link)
I concur.
elfin: image: dreamsheep with fringe apple (dreamsheep.fringe)

[personal profile] elfin 2011-03-30 12:04 am (UTC)(link)
Huh, that's really interesting. I've never thought of coding in those terms. Thanks for posting that.
xtina: (Default)

[personal profile] xtina 2011-03-30 06:13 am (UTC)(link)
This entirely.
ironed_orchid: pin up girl reading kant (Default)

[personal profile] ironed_orchid 2011-03-30 11:15 am (UTC)(link)
Well you just introduced me to the term, and it makes so much sense.
sam_gardener: Text: Dreamwith. dreamsheep inside a stargate and holding a stargate (sg_dreamsheep submerged)

[personal profile] sam_gardener 2011-03-30 08:06 pm (UTC)(link)
Yes! This is the first time I've heard (noticed?) this term and it describes perfectly the ominous feelings I've been having recently. I am currently creating technical debt much to quickly and it's been giving me nightmares. It's nice to have the right name for it.

Also - great article, thank you for taking the time to write in such detail!
torachan: arale from dr slump with a huge grin on her face (arale)

[personal profile] torachan 2011-03-30 12:11 am (UTC)(link)
Thank you for writing this up, and thank you guys for doing all this work that will allow DW to be even awesomer. :D
anonymous_sibyl: a close up of ice covered branches. (Default)

[personal profile] anonymous_sibyl 2011-03-30 12:16 am (UTC)(link)
This is fascinating. Thanks for explaining it.
azurelunatic: A glittery black pin badge with a blue holographic star in the middle. (Default)

[personal profile] azurelunatic 2011-03-30 12:17 am (UTC)(link)
Thank you for laying it all down like this.
jadey: greyscale a woman's face (ani difranco) eyes upward  (Default)

[personal profile] jadey 2011-03-30 12:20 am (UTC)(link)
Best Internet history lesson EVER. I'm embarrassed, but I had no idea about any of that.
foxfirefey: A guy looking ridiculous by doing a fashionable posing with a mouse, slinging the cord over his shoulders. (geek)

[personal profile] foxfirefey 2011-03-30 12:22 am (UTC)(link)
There's nothing to be embarrassed about!
alexseanchai: Purple lightning (Default)

[personal profile] alexseanchai 2011-03-30 12:55 am (UTC)(link)
Seconded.
chagrined: Marvel comics: zombie!Spider-Man, holding playing cards, saying "Brains?" (brains?)

[personal profile] chagrined 2011-03-30 12:22 am (UTC)(link)
This was a cool writeup, thanks.
john: An unclothed man's groin; no actual nudity. (naked)

[personal profile] john 2011-03-30 12:31 am (UTC)(link)
This is the most interesting thing I have read all week. <3
ilyena_sylph: picture of Labyrinth!faerie with 'careful, i bite' as text (Default)

[personal profile] ilyena_sylph 2011-03-30 12:57 am (UTC)(link)
This is totally awesome, and I am glad to get to hear it.

Also, I may be the weird one, but I like a high degree of stability, this is one of the reasons I liked LJ before SA and now am madly in love with Dreamwidth.

I *don't like change* -- NB, I am in love with the idea of the new update page, but this is probably because y'all had the decency to *ask!* -- and I feel stable here.
ilyena_sylph: picture of Labyrinth!faerie with 'careful, i bite' as text (Default)

[personal profile] ilyena_sylph 2011-03-30 02:12 am (UTC)(link)
*grins cheerfully* I think the DW-news posts do prove that I'm not alone, but if you're getting whining about that DW isn't releasing enough new stuff, data-points to the contrary are sometimes useful?

(heh. I miss having a decent multi-chat client, I really do.) And yeah, I agree with you. Things have to change and improve or they die, and I want DW to thrive. I love your a) b) and c) of looking at it, and they make all the difference in the world.

Heh. I am one of those people that totally doesn't see them, I think. *nods* I do get what you mean there, and in case it wasn't clear, I appreciate the background detail you went into about everything a lot.
ilyena_sylph: picture of Labyrinth!faerie with 'careful, i bite' as text (Default)

[personal profile] ilyena_sylph 2011-03-30 02:39 am (UTC)(link)
I would be fascinated by this post!

(I miss the ability to post media somewhere I actually have some faith in, but I don't miss scrapbook, quite definitely!)
john: Various candles, in multicoloured jars, under trees in the evening (nz:hobbits)

[personal profile] john 2011-03-30 05:20 am (UTC)(link)
Yeah, I only ever used Scrapbook as an "email address from my phone" posting method anyway.
dhae_knight: (Dreamsheep Denmark)

[personal profile] dhae_knight 2011-03-30 09:38 am (UTC)(link)
Agreed, Scrapbook is *atrocious*. I used it to put up mood icons on my LJ, and it took me three days and reading several dozens of how-to pages to get it done. :-/
pne: A picture of a plush toy, halfway between a duck and a platypus, with a green body and a yellow bill and feet. (Default)

[personal profile] pne 2011-03-30 11:12 am (UTC)(link)
Now I'm wondering whether the Singles database was also horrible back-end code and/or horrible UI. I think that's another feature that nobody much misses - not least because probably next to no one knew about it on LJ in the first place!
pne: A picture of a plush toy, halfway between a duck and a platypus, with a green body and a yellow bill and feet. (Default)

[personal profile] pne 2011-03-31 08:13 am (UTC)(link)
d) I mean, really, wtf?

Haha, indeed! So not LiveJournal's "core competency".

Now I wonder when you'll have paid back enough technical debt and set up the infrastructure so that you can bring back the Nudge feature properly.

(I kid!)
pauamma: Cartooney crab holding drink (Default)

[personal profile] pauamma 2011-03-31 12:24 pm (UTC)(link)
There is a running joke that it was informally called the "Brad can't get a date" feature.
sophie: A cartoon-like representation of a girl standing on a hill, with brown hair, blue eyes, a flowery top, and blue skirt. ☀ (Default)

[personal profile] sophie 2011-03-31 12:55 am (UTC)(link)
It's interesting that you mention Singles, because LiveJournal just ripped it out today!
pauamma: Cartooney crab holding drink (Default)

[personal profile] pauamma 2011-03-30 05:44 pm (UTC)(link)
Scrapbook makes the translation system look almost good in comparison. (And when you look at the Scrapbook backend - and the way it communicates with blogging systems - you can strike out the "almost".)
dglenn: Lego-ish figure in blue dress, with beard and breasts, holding sword and electric guitar (lego-blue)

[personal profile] dglenn 2011-03-30 06:44 am (UTC)(link)
"hell, I'm still using a multichat client that was EOL'd in like 2005 or so, because I hate the user interface of every alternate client I've tried with the burning passion of a thousand suns"

Uh, yeah ... that's about 4/5 of the reason that I'm still using a MUA that's a direct descendent of /bin/mail. (Though I was indeed happy to find a newer version a few years ago, that adds attachment-handling while retaining my comfortable-like-old-jeans mailx UI.) I do have actual technical requirements for which it's the best solution, as well -- and those might well be enough on their own -- but usually before I even get as far as looking at those, I'm already reacting to a proposed MUA's UI with, "Nuh-uh, that's gonna drive me up a tree if I try to use it for more than ten minutes; gimme back mail/Mail/mailx/nail/Heirloom-mail right now."

Change can be really, really good (and I'm sure there can be a better -- even a better-for-me -- mail UI) but I'm not afraid of being old-fashioned when the newer things so far either suck outright, or fail to fit me well.
dharma_slut: They call me Mister CottonTail (Default)

Funny you should mention it!

[personal profile] dharma_slut 2011-03-30 12:58 am (UTC)(link)
I've spent a month looking at shopping carts to replace my ancient, delaminating, often hand-coded OSCommerce site. When I look at the primitive state of PHP language back then I must have been hitting it with rocks and grunting at it to make it work. I can't do that now. And I have experienced that thing with the incomprehensible code that I myself wrote once on a time.

(I'm working with a developer to create a craftsperson's cart for wordpress)

Don't do those social network things, please? If there were one thing I would ask you for-- it would be to implement a page or series system for the fiction writers, so that they could link chapter posts together outside of the chronology. If you did that, I would totally transfer my writer's blog and archive to DW. (Are URLS implemented yet?)
dharma_slut: They call me Mister CottonTail (Default)

Re: Funny you should mention it!

[personal profile] dharma_slut 2011-03-30 02:01 am (UTC)(link)
Oh, sorry-- although pretty URLS would be lovely, I should have said Domain referrals. So my domain http://stellaomega.com could come straight here. You might have that already-- I've forgotten to check.

Index pages as you've described them would be wonderful! Like a cover for every book! The other thing would be a link to the previous and next page as someone reads through in the series, and maybe a way to enter the post into the series order, although not so many folk write as out-of-order as I do-- although you do talk about being able to move entries up and down in the list.
dharma_slut: They call me Mister CottonTail (Default)

Re: Funny you should mention it!

[personal profile] dharma_slut 2011-03-30 02:03 am (UTC)(link)
I shall wait patiently for it then :)
niqaeli: cat with arizona flag in the background (Default)

Re: Funny you should mention it!

[personal profile] niqaeli 2011-03-30 02:03 am (UTC)(link)
I believe you should be able to setup domain aliasing under Manage Account, on the first tab (Account). Here is a direct link to the setup page.
pauamma: Cartooney crab holding drink (Default)

Re: Funny you should mention it!

[personal profile] pauamma 2011-03-31 12:38 pm (UTC)(link)
Unless the implementation changed significantly since it was inherited from LiveJournal, it has some restrictions, like not showing your restricted entries, or any restricted entries on your reading page. Removing those is feasible, but it requires care making sure the right cookies are set for the right domains and that no one is inadvertently given access to entries they shouldn't be able to see.
matgb: Artwork of 19th century upper class anarchist, text: MatGB (Default)

Re: Funny you should mention it!

[personal profile] matgb 2011-03-30 02:34 am (UTC)(link)
I had an idea re pretty URLs that may or may not be good, but as you've mentioned it and I'm here, a hopefully possibly simple fix that would work for what we need it for.

Currently, every entry gets ENTRYNUMBER.html

How about dumping .html and putting in a / instead. Get the servers to ignore anything after the / and just send you to the entry. Get modrewrite or whatever it is to display the current entry title after the / with hyphens.

So this entry would be

http://denise.dreamwidth.org/57248/technical-debt-and-the-making-of-payments-on-it/

Idea nicked from a few newspaper websites that do similar. Solves the title editing problem, probably solves a whole bunch of redirect issues, and means those that don't want pretty urls get entry addresses that can be converted back to them easily. If a paid feature, solves what happens if they revert to free.

However, I've no idea how complex to code it it would be, and there may be security holes (other than the obvious pranksters rewriting links).

(FWIW, domain mapping that works properly is near the top of my personal wishlist, but UI for commenting and off site logging in is above that, and pretty URLs slightly below)
fu: Close-up of Fu, bringing a scoop of water to her mouth (Default)

Re: Funny you should mention it!

[personal profile] fu 2011-03-30 03:34 am (UTC)(link)
Oh hmm that's really neat and means we wouldn't have to worry about caching.

But I also recall seeing links gleefully passed around that troll the news sites, when the pretty part is ignored completely.

So perhaps a compromise: use the post number to look up the entry, and then double-check against a saved version of the URL slug or something of the sort? Then you couldn't have http://denise.dreamwidth.org/57428/dw-just-bought-out-by-6a (or mis-titles along that line)
matgb: Artwork of 19th century upper class anarchist, text: MatGB (Default)

Re: Funny you should mention it!

[personal profile] matgb 2011-03-30 03:25 pm (UTC)(link)
Guess you could, but it would mean either storing multiple versions or having problems if someone links to a post that then gets edited.

As long as whatever the page landed on displays the URL correctly and there's a canonical URL, then it'll do what's needed for search and usability purposes, and while it's likely there will be a small amount of abuse, I think that matters not a lot.

But, if abuse and similar are a lot more concerned about it then perhaps might be worth doing.
adalger: Earthrise as seen from the moon, captured on camera by the crew of Apollo 16 (Default)

Re: Funny you should mention it!

[personal profile] adalger 2011-03-30 06:12 pm (UTC)(link)
Why couldn't you have <a href="http://denise.dreamwidth.org/57248.html>http://denise.dreamwidth.org/57428/dw-just-bought-out-by-6a</a> for the same issue? Is there really enough of a qualitative difference for this concern to be meaningful?
dharma_slut: They call me Mister CottonTail (Default)

Re: Funny you should mention it!

[personal profile] dharma_slut 2011-03-30 09:02 pm (UTC)(link)
How does wordpress do it?
pauamma: Cartooney crab holding drink (Default)

Re: Funny you should mention it!

[personal profile] pauamma 2011-03-31 12:43 pm (UTC)(link)
IIRC, the software can do it in a number of different ways, all incompatible, at the blog owner's option. I'm not sure what the hosted service does.
jumpuphigh: Pigeon with text "jumpuphigh" (Default)

Re: Funny you should mention it!

[personal profile] jumpuphigh 2011-03-30 06:36 am (UTC)(link)
I totally started using Tumblr because posting a picture there is so darn easy.
finch: (Default)

Re: Funny you should mention it!

[personal profile] finch 2011-03-31 01:16 am (UTC)(link)
I'm actually glad to hear you're studying Tumblr and its ilk. There are a lot of things I like about Tumblr but the lack of comments and the resulting need to wade through reblogs in reverse chronological order to find - if you're lucky - the post everyone is replying to drives me mad. (I've seen people roleplaying on Tumblr and I simply have no idea how they can stand it.) I'd love to have more of the simple-upload-post functionality of Tumblr that retains the actual-working-comment-system of DW and does it intelligently.
musyc: Silver flute resting diagonally across sheet music (Default)

Re: Funny you should mention it!

[personal profile] musyc 2011-03-31 02:01 am (UTC)(link)
Seconding on the comment system re: Tumblr. Tracking back to an original poster is a NIGHTMARE, and the askbox/"someone replied to your post" system is completely useless for conversations more than three comments long.
sphinx_child: (Default)

Re: Funny you should mention it!

[personal profile] sphinx_child 2011-04-06 07:25 pm (UTC)(link)
Wait. What? RP on TUMBLR? How does that...

I think I need a lie down now. Why would you DO that to yourself?
finch: (Default)

Re: Funny you should mention it!

[personal profile] finch 2011-04-06 07:47 pm (UTC)(link)
I don't get it. I mean, I try not to knock it - I've done twitter roleplay - but I can tell that it wouldn't be much fun for me, that's for sure.
roadrunnertwice: Davesprite from Homestuck, Mr. Orange Creamsicles hisself (Homestuck - Davesprite)

Re: Funny you should mention it!

[personal profile] roadrunnertwice 2011-04-06 06:04 am (UTC)(link)
I'm studying Tumblr pretty closely right now, for instance…

I am super glad to hear that! I think Tumblr threw a certain amount of baby out with the bathwater, but damn do they have a sparkly tub. (And its explosive growth has convinced me that social journaling is still an almost-mainstream thing; it's just that folk demand that it keep up with their other online tools.)
st_aurafina: The dreamwidth sheep, sleeping peacefully (Dreamwidth: sheep)

[personal profile] st_aurafina 2011-03-30 01:06 am (UTC)(link)
Wow, this is just fascinating - I had no idea all that was going on beneath the surface of my dreamwidth journal! Thank you for explaining it so clearly.
kateshort: (circuit_breaker)

[personal profile] kateshort 2011-03-30 01:13 am (UTC)(link)
Awesome explanation. Thank you so much for taking the time to lay all of this out!

I'm sure you'll get some TL;DR responses, but this needed to be long, and I'm glad I read it.
elucidate_this: (Default)

[personal profile] elucidate_this 2011-03-30 01:16 am (UTC)(link)
This was really fascinating! Thank you for taking the time to write it all up.
happydork: A graph-theoretic tree in the shape of a dog, with the caption "Tree (with bark)" (Default)

[personal profile] happydork 2011-03-30 01:25 am (UTC)(link)
This is fascinating! Thank you for explaining it so clearly.
emceeaich: A close-up of a pair of cats-eye glasses (Default)

[personal profile] emceeaich 2011-03-30 01:35 am (UTC)(link)
I had no idea about the Apache and Perl version dependencies that you had to pay your way out of.

My hats off to the devs.

the problems LJ solved back then are universal to any high-load system, and the solutions they/we came up with are still in use today

Yes. So many facebook fanboys need to be slapped upside the head with a copy of memcached.

Meanwhile, at the job I'm starting next week, I'm inheriting a 10 year old HTML:Mason deployment! Woo!
niqaeli: cat with arizona flag in the background (my kitty brethren)

[personal profile] niqaeli 2011-03-30 01:58 am (UTC)(link)
You don't need more than one act of true genius to be a genius. And Brad had more than one act of genius on the LJ codebase.

And then, you know, there was Scrapbook which I always thought a deranged, disoriented weasel must have laid out the UI.

But I still admire those acts of genius.
pauamma: Cartooney crab holding drink (Default)

[personal profile] pauamma 2011-03-30 06:40 pm (UTC)(link)
Speaking of Brad, this changelog message still amuses me beyond reason.
sophie: A cartoon-like representation of a girl standing on a hill, with brown hair, blue eyes, a flowery top, and blue skirt. ☀ (Default)

[personal profile] sophie 2011-03-31 10:10 pm (UTC)(link)
I never realised there was an older QDB! If the old database for it is lying around on someone's hard disk, you may want to let [livejournal.com profile] idonotlikepeas know so he can import it into today's LiveJournal QDB.
pauamma: Cartooney crab holding drink (Default)

[personal profile] pauamma 2011-04-07 06:12 pm (UTC)(link)
Maybe asciident does?
asciident: (Default)

[personal profile] asciident 2013-11-25 10:35 am (UTC)(link)
I realize this reply is literally about 2.5 years too late, but just to confirm that I do not have any old QDBs aside from the IRC RP campaign, Missing Persons, D ran several years ago. I have gone through more hardware than I care to name and a cross-country move and lost a lot of data along the way.
yvi: Kaylee half-smiling, looking very pretty (Default)

[personal profile] yvi 2011-04-07 04:30 pm (UTC)(link)
I laughed so loud I scared the cats.
pauamma: Cartooney crab holding drink (Default)

[personal profile] pauamma 2011-04-07 06:13 pm (UTC)(link)
My work here is done.
niqaeli: cat with arizona flag in the background (Default)

[personal profile] niqaeli 2011-03-30 01:54 am (UTC)(link)
None of this is news to me because I am a Speshul Fucking Snowflake who learns by osmosis and absorbed a lot of these concepts while very young from my father, but it's always nice to see someone lay it out clearly for the layperson. This is well written and communicates the concept of technical debt pretty simply for people who have no desire or need to understand software engineering in any kind of depth.

I've been really excited to see all the refactoring and modernising of code and I had kind of forgotten that it might not look very exciting to most people.

Anyway! Yay this entry and I'm glad you're going to go into this in a little more detail in the next news update than have gone into past updates.
aedifica: Me looking down at laptop (off screen).  Short hair. (Default)

[personal profile] aedifica 2011-03-30 01:58 am (UTC)(link)
It was just this morning I was reading about "mature software environments" in a novel--and the description in the novel made it sound like an environment where they never did pay off their technical debt, just layered over it with other stuff. :-)
owl: Stylized barn owl (Default)

[personal profile] owl 2011-03-30 03:52 pm (UTC)(link)
That is so true! My employers are attempting to develop a 'service oriented architecture' across the organisation, and it's like the layers of an onion: shiny RIA and mobile clients/UIs (2010's tech), then a lot of Java and webservices (2000's tech, buffed up by IBM), then as you dig deeper you find the C++ code from the 1980s and 1990s, and then at the bottom it's all still COBOL.
aedifica: Me looking down at laptop (off screen).  Short hair. (Default)

[personal profile] aedifica 2011-03-30 04:34 pm (UTC)(link)
And I like onions so much better in cooking than in code! *grin* So yay DW for peeling the onion... or something.
pauamma: Cartooney crab holding drink (Default)

[personal profile] pauamma 2011-03-30 06:57 pm (UTC)(link)
I'm sure if you dig in deeper you'll find CICS command-level code. Maybe even backward combatibility stuff for TCAM.
owl: Stylized barn owl (Default)

[personal profile] owl 2011-04-02 03:03 pm (UTC)(link)
Ha. Ha. Funny now you mention CICS....
pauamma: Cartooney crab holding drink (Default)

[personal profile] pauamma 2011-04-02 03:10 pm (UTC)(link)
I really wish I'd guessed wrong. :-(
reddragdiva: (Default)

[personal profile] reddragdiva 2011-04-06 11:20 am (UTC)(link)
This is how it's really done in the commercial world. Layers on layers on layers, until the system is in so much debt they declare it bankrupt and roll out a new version. Repeat.

Commercial code is generally much worse than open source. The reason many companies don't release their code as open source is shame.
reddragdiva: (Default)

[personal profile] reddragdiva 2011-04-06 11:30 am (UTC)(link)
I have a friend who is a Visual Basic in Excel programmer. He consults to banks for a frickin' fortune.

Most of the millions and billions of dollars flowing around in finance? Yeah, Excel macros.
cmshaw: Leverage: when Hardison pushes the button, everything goes boom (Geeks bring the firepower)

[personal profile] cmshaw 2011-03-30 02:26 am (UTC)(link)
Code that is brilliant, flawless, and crystal clear when you write it slowly morphs into idiotic, bug-ridden, and clear as mud over time. This is a well-known process. I suspect pixies in the source code repository, working their anti-magic while nobody's looking.

Yes! Darn them. They get into everything!

This is why I like writing flippant comments in my code. It reminds me of my mindset at the time I was first writing it and helps me look at things the right way to pick up and work on it again. Yes. That's my story and I'm sticking to it.
dglenn: Me in poufy shirt, kilt, and Darth Vader mask, playing a bouzouki (vader)

[personal profile] dglenn 2011-03-30 07:00 am (UTC)(link)
Ayup. It's a big part of the reason the project I'm currently working on is 51% comments, 17% whitespace, 34% code (in the source/script files; not counting the explicitly-documentation files). Okay, part of it is that I babble when I get tired, but most of it is that I want whoever tries to modify it next to be able to make sense of it without the ability to read my mind, so it gets edited instead of ripped-and-replaced ... especially if that next person is just me again in a year. There are fewer "WTF were they thinking?" moments when the comments explain what you were thinking, why you went one way instead of another.

Unless the code pixies f$#% with the comments as well, of course, in which case, well, I'm screwed.
pauamma: Cartooney crab holding drink (Default)

[personal profile] pauamma 2011-03-30 07:50 pm (UTC)(link)
*thinks of some developers he worked with* "as though"? (At one of my fiorst few jobs, there was a "watch out for flying terminals" sign. That wasn't a hoax or a joke.)
sophie: A cartoon-like representation of a girl standing on a hill, with brown hair, blue eyes, a flowery top, and blue skirt. ☀ (Default)

[personal profile] sophie 2011-03-31 10:13 pm (UTC)(link)
I have actually apologised to the next developer in comments because of this!
ct: SG-1: Sam and a laptop. Text: The one and only true ship! (SG1: sam otp)

[personal profile] ct 2011-03-30 07:19 am (UTC)(link)
I don't know if it's pixies, but something's gotta be responsible for all the comments I've seen at work that read "If the meaning of [field name] isn't clear, there really should be more of a description here...".

They're automatically generated comments, yes. But if you figure that people are only likely to read the comments if the field name wasn't enough to explain its purpose, having that as the default message is just cruel.
vatine: Generated with some CL code and a hand-designed blackletter font (Default)

[personal profile] vatine 2011-03-30 10:31 am (UTC)(link)
Somewhere in my code at home, there is a function of (roughly) 50 lines of code (excluding blank lines) that follows a (roughly) 45-line comment, explaining what it does (not how, that's what the code is there for). It's an implementation of getpwent(3) for a /etc/passwd replacement using a directory structure (linear searches get So Slow when you hit 30k+ users, you know). It's all horrendously stateful and stuff, doing Plain Nasty Things in order to actually work.
jld: (Default)

[personal profile] jld 2011-03-30 02:41 am (UTC)(link)
Thanks for writing that up; it's a really nice explanation of a lot of things.

(Also, fun fact: despite reading Brad's journal back in the pre-6A days, I did not know about the Apache version thing. OM*G*.)
sraun: portrait (Default)

[personal profile] sraun 2011-03-30 02:46 am (UTC)(link)
My father has the worst 'code come back to haunt me' story I've heard yet.

He worked for the electric power company in Akron, OH in the late 1950s, where he wrote some Fortran to help analyze power distribution networks. After about five years, he left that job and moved to Duluth, MN, where he went to work for a different electric power company.

Fast-forward to sometime in the mid-80s. Dad is working on distribution planning (how much power are we going to need to provide where in 5, 10, or 20 years, and what do we have to do to our network to meet those needs), and needs to some specialized network analysis. He goes rummaging into the software library, and finds a program that should do just what he wants, with a little tweaking.

The specific section he needed to modify was about 30 years old. According to the changelog, it was untouched since it had been written. He looked at it, went 'I remember writing this code, and I haven't the faintest idea how it works'. He tore the whole section out and re-wrote it from scratch.

Yeah, I intuitively understood the concept of technical debt pretty much the moment you used it.
jenrose: (FilkerKazoo)

[personal profile] jenrose 2011-03-30 04:30 am (UTC)(link)
Lords of Fortran and Cobol. Or as Frank Hayes puts it, "Back when Fortran was not even Threetran!"
pauamma: Cartooney crab holding drink (Default)

[personal profile] pauamma 2011-03-30 08:08 pm (UTC)(link)
Looks like the kind of code this license is for...
sraun: portrait (Default)

[personal profile] sraun 2011-03-30 11:58 pm (UTC)(link)
I don't know about that - it apparently went through lots of people between the time he wrote it and the time he needed it. It was just that that one routine never got touched.

It is also true that he was only a couple of years out of college when he wrote it in the first place.
ariestess: (Default)

[personal profile] ariestess 2011-03-30 03:21 am (UTC)(link)
Thank you for this. I haven't had any complaints re: DW, but it's nice [and fascinating] to read how some of this stuff comes about.
j00j: rainbow over east berlin plattenbau apartments (Default)

[personal profile] j00j 2011-03-30 03:33 am (UTC)(link)
Thanks for this explanation-- it was a really helpful behind the scenes look at development of projects like this. Good to think about when I see how much technical debt we're having to chip away at in my day job as well.
poulpette: Stick-figure of a smiling head, raising heart pompoms. Bottom half says YAY!!! (Misc - devs = awesome)

[personal profile] poulpette 2011-03-30 03:40 am (UTC)(link)
This has been a most enlightening and interesting read. Thank you for writing this [staff profile] denise, and for the clarity of your explanations. Kudos <3
jecook: (Default)

[personal profile] jecook 2011-03-30 04:06 am (UTC)(link)
Good lord, I had no clue that the old LJ code was that kludgey. Odd at times, yes. full of 'quirks', yes- any major application is going to have those.

But to only work with one version of Perl and Apache? OUCH.
laitaine: (flowers - pink tulip)

[personal profile] laitaine 2011-03-30 07:50 am (UTC)(link)
This was really interesting, and made me appreciate even more what you guys are doing here. ♥
lobelia321: (Default)

[personal profile] lobelia321 2011-08-08 05:52 pm (UTC)(link)
This is so fascinating. Thank you so much for explaining what to me is a totally arcane world of gobbledygook. (*is software-illiterate*)

Page 1 of 3

<< [1] [2] [3] >>