RSS Feed

The Design of Software (CLOSED)

I am currently working on a project that has little to no design. This process was conceived because the system is very large and it would take a significant amount of time to complete a comprehensive design. I have found this to be extremely challenging. Anyone else have similar experience or advice?

In this field, we call that experience "software development."  :(

"This process was conceived because the system is very large and it would take a significant amount of time to complete a comprehensive design."

This is possibly the worst reason to not have a plan. If the system is that complex, does anyone think that the accompanying code will be less so? Does wondering around in a really big, dark forest without a map make more sense because it's big?

I'm currently doing something similar: "they" sold a significant expansion of our product to a big & highly visible customer.  Said expansion _does_ give us better coverage of our market's needs.  BUT....they sold a delivery date without even rough requirements, let alone any kind of detailed specs.  Now we're pushing three months late on the promised delivery & the requirements are still being worked out.

Point in my favor: I did a risk assessment up-front & told my boss that the lack of requirements/specs and consequent lack of developer input on a viable delivery date made it almost certain that we'd be late no matter how simple the requirements were.

Doesn't buy back anybody's weekends, though, and one of the developers has kicked off several builds in the midnight-to-four-AM range.

Almost any kind of up-front sizing or planning is better than none.  The bigger the project, the more important it is.  Even for what I know of XP (next to nothing), there's _some_ kind of sizing effort for each iteration.

> Even for what I know of XP (next to nothing), there's _some_ kind of sizing effort for each iteration.

From what I remember reading, some of the aspects of XP include:

* Short iterations
* Two parts to each iteration: first define the requirements (the to-do list for this iteration), and then implement them (the "sprint")
* Have customer input on the requirements (for this iteration)

It is possible to work on big project without a comprehensive up-front design - in fact, trying to design ALL of a complex system up-front can be an endless exercise.

However, you do need an overview of the requirements, a priority list of which ones to tackle first, and detailed requirements and a realistic schedule for the first release.

Some of you guys get designs?!?!?! Aw c'mon, next thing you're going to be telling me you get compilers too!!

We like analogies from the construction industry, not that they don't have snafus as well, but here it is.

You could imagine building a dog house without a plan, or just scribble a few dimensions on a corner of your shop table, but would you build a 4,000 foot suspension bridge without a plan?

Now things like Extreme Programming are intended to reduce up-front planning, but the planning must be done somewhere.

And at the end of the project is the most painful, expensive, and lengthy time to be doing that planning.

>>Some of you guys get designs?
Well, sorta.  We know it has to run on Windows & numbers are involved somewhere. <g>

"Anyone else have similar experience or advice?"

I would start doing the sorts of things described in Fowler's _Refactoring_ book and/or the _Refactoring to Patterns_ book by Kerievsky. 

The subtitle of Fowler's book is "Improving the Design of Existing Code", and that's what refactoring is all about.  Take a piece of code that has poor design, refactor to make design better, wash, repeat. 

With a big project you'll be doing lots of different refactorings, gradually over time if done right an overarching design will  appear. 

An approach like this is why a lot of people (e.g.,  Joel: http://www.joelonsoftware.com/articles/fog0000000069.html ) recommend against rewriting and instead advise you to revise the existing _working_ code.

Virtually all software is built in this way. Virtually all articles on software development are about this topic. Virtually all software products fail or at least don't meet expectations. This is the fundamental problem in software engineering, and as Fred Brooks says, "there are no silver bullets". (Look it up)

However, there is something you can do to help out. It's called "discipline". Force yourself to design and use a anal-rentitive coding convention, even though the language doesn't actually care about spacing and variable names. Make every file have a consistent header. Refactor mercilessly. Meet every point in The Joel Test, no matter how much it seems like they don't contribute to actual coding.

This won't solve all your problems, but it will help make the experience less unpleasant. Good luck.

interface Client
{
  Requirement requirement { get; }
}

class Developer
{
 //data
 Client client;
 Software software;
 List<Requirement> requirements;
 List<Bug> bugs;
 List<Test> tests;
 //public accessors
 public void add(Bug bug) { bugs.Add(bug); }
 public void add(Requirement requirement) { requirements.Add(requirement); }
 //active method
 active void run()
 {
  for(;;)
  {
  //fix any bugs before continuing
  debug();
  //implement the next requirement
  Requirement requirement = requirements.Pop();
  if (requirement == null)
  {
    //get new requirement from client
    requirement = client.requirement;
  }
  implement(requirement);
  }
 }
 //private helpers
 void debug()
 {
  foreach (Bug in bugs)
  {
  implement(new Requirement(bug));
  }
 }
 void runTests()
 {
  foreach(Test test in tests)
  {
  if (!software.run(test))
  {
    add(new Bug(test));
  }
  }
 }
 void implement(Requirement requirement)
 {
  List<Plan> plans = inspect(software);
  foreach (Plan planned in plans)
  {
  software.change(planned.codeChange);
  foreach (Test test in planned.testCases)
  {
    tests.Add(test);
  }
  runTests();
  debug(); //caution: recursive if a bug fix causes a bug
  }
 }
}

> List<Plan> plans = inspect(software);

... should be "List<Plan> plans = createPlans(software,requirement);"

The point is, I guess, that you can work with even one requirement.

Not shown above is developing plans that are based on several requirements simultaneously.

all software, when it has been changed enough, eventually looks like it doesn't have a design. one of the big problems in software is that it is quite hard for the architecture to be understood without external documentation. and of course external documentation is usually out of date and therefore  almost completely useless.

One of the rules of thumb from _Systemantics_ is, "A complex system that works is invariably found to have evolved from a simple system that worked."

I understand Requirements Up Front (with the understanding that they'll change), but I never understood Design Up Front as being a separate act from coding.  "Don't touch that keyboard yet!  You haven't spent enough time in the corner thinking really really hard about it yet!"  But as soon as I have a really slick Design Up Front, I'm already finding flaws with it *moments* after I start coding.

Why can't we just admit that there's more than one kind of coding, and that coding-as-design is the first tool we should pick out of our toolbox?  Just because 90% of you people write spaghetti doesn't mean that coding-as-design produces spaghetti - anyone who thinks so is confusing correlation with causation.  Amateurs produce crap whether they think really hard about it Up Front or not, let the rest of us *actually* design using the most relevant tools, please.

Software development is really hard.  But designing without coding is designing with a blindfold that keeps you from interacting with *the most important thing* - the code.

>  But designing without coding is designing with
> a blindfold that keeps you from interacting with
> *the most important thing"

It seems the word design gets tossed around at different scales of a project like it's all the same thing.

It's design to pick tools, an OS, and a language. Should everyone just pick different ones of these?

It's design to say it's web app or thick app. Should everyone on a project just pick different ones of these?

It's design to say we are going to use clustering. Should everyone just go there own way.

It's design to say I'll use a link list. Should someone really plan on using a link list?

There are designs, and there are specifications.
There are user designs, and there are technical specifications.
There are technical designs, and there are user specifications.
Careful.

Sometimes I find the two terms are used interchangably, but they are not.  It is possible to work on a system without a master design, though that is asking for trouble.  The agile methods call for "throwing one away" and/or lots of refactoring, which always seems to be forgotten.

I will *never* again work on a system without functional user specs.  That is always a burn, regardless of how "special" the situation is.  Every time I've started without user specs the project has been a disaster, even with just one programmer.

You need a functional user specification design document.

Interestingly enough, if you look at a set of architectural blueprints for a house, the first "release" is surprisingly vague and usually doesn't mention plumbing, electricity, ventilation or any of that stuff. Just a basic idea of where the walls go, where the staircase goes (if any), and what architectural style the finished house should resemble.

I don't think it's bad to code without a design as most of us interpret the question. I do think it's bad to code without some sense of the overall business goals you're trying to achieve. It's a disaster if you can't describe what your project should do - when finished - to an innocent bystander.

I agree with TheDavid... you need to have enough of a design/requirements that you can build the general house, but you don't want to figure out where every last wire goes. I've worked on projects where we tried to figure everything out in advance, and its just a nightmare... I also find problems with the design *moments* after starting coding.

I actually wrote a post on this a while back... how to write requirements without going overboard. Rule #1: Keep It Simple, Stupid! Or else nobody will pay attention.

http://www.startupcto.com/articles/the_story_of_a_paper_clip_writing_good_functional_requirements

"Coding Without A Design" is needed if you are building a "product".

The two posts that hit the mark here are from 'dwayne' and 'dot for this one'. Simple projects need some design - complex projects need more.

If you are working on a complex project where "we aren't doing any design because there isn't time" then I recommend a book that will help you understand the situation - "Deathmarch" by Edward Yourdon.

The typical way to handle large projects with indeterminate specs is to design just an application framework as thin as possible, then later design modules that plug into that as the app evolves. Each module has is its own internal design and doesn't depend on other modules, only that thin framework "glue."

"...doesn't depend on other modules..."

In complex system you want to reuse code, so you end up having modules depend on other modules. Simply, no other ways...

> In complex system you want to reuse code, so you end up having modules depend on other modules. Simply, no other ways...

Instead, you can have modules depend on abstract interfaces.

You might have a bottom, library or O/S-abstraction layer on which all module implementations depend, but apart from that the modules (plug-ins) might communicate via abstract interfaces.

Great mirth!!  The irony...

"...This process was conceived because the system is very large..."

I guess I forgot that you only need to architect small projects - foolish me!

I'm sorry that I can't give you any advice because I've never been in such a ridiculous situation.  But it sounds like "Deathmarch" by Yourdon is just over the horizon for you and your team.

Print all these posts out if you're a "I told you so" type.

It really seems to me that most of the people that have contributed to this WANT and EXPECT to live in an ideal world.
The fact is, almost everyone responding is doing so from the perspective of a developer. Hello...how come you developers (including myself) have a job, even it that job is solving world poverty, finding a cure for cancer, and solving the problem of climate change? It is because some wanker from marketing has sold a concept (yes, even with one hand under the table) to the customer.
You have to find a way to deliver. Where is the competitive edge if every thing is specified and designed to the "nth degree"? What will distinguish your "fully specified" solution from the "fully specified" solution from your competitor. Probably price (predicated on the assumption that your company and your competitors both possess people and processes that enable a product that works to be developed). That means costs and/or time. Costs (or the reduction of same) typically mean less upfront documentation (read specifications) because this is not something the customer is excited about when you have your first contract progress review.
I am 3 years into a defence project after 20 years in the commercial world, and I find the emphasis on (complete) design and specification upfront an unbelievable overhead and burden on productivity. Anyone heard of the spiral methodology? Anyone every worked on a project so large and complex, that to fully design and specify the product up-front means it is irrelevant by the time development starts, let alone is complete?
I have always worked with a design or specification of some kind. The fact that is has sometimes been nothing more than the aforementioned "marketing wanker's" printout from an electronic whiteboard does not lessen its value.
Much depends on the experience and ability of your developers. If your karma is good, you can trust them to take it and run with it. If not, the requirements specification for your requirements specification will range from what keys to press in which order, to a loose textual description in pseudo-code.
You must have something to start with.
Where do you fit into this? Are you a team lead, or a member of the team? This will, in many ways, determine what your response to the problem should be.
Finally, after a lot of time doing this sort of thing, my take on things:
1) Treat it as a prototype, with the foreknowlege that it will eventually become a product. This saves lots of wasted time trying to get small things right the first time around.
2) The most critical things to prototype (and thus get right) are interfaces, roughly in the order of machine-to-machine, process-to-process (which could be thread-to-thread or class-to-class, etc) then machine-to-user.
3) Complete documentation (including design and specifications) CAN come after coding. There is no absolute single best or "solve all problems way to do this. If there was, the engineers would have won out a long time ago, programmers would be redundant, and all code would be forward generated by software design applications. Every application would simply be a construct consisting of tried-and-true bricks and beams put together in a way that someone has proven works before.
THIS IS NOT THE WAY THE REAL WORLD WORKS.