A few months ago, I was on a flight to Boston for two job interviews. I was reflecting on the art and practice of software engineering, and decided to jot down my thoughts. I wrote the first bit on my way out, the second bit (mostly documentation) on my way back to West Virginia two days later, but I never really finished it.
Rather than have this stuff laying about as an unfinished draft that nobody ever sees, I've decided to post what I've already written and see if it garners any real interest. If I get sufficiently inspired, I'll continue it.
- Simple is good. Simplistic is bad. This means that you want your design to be as simple as possible, but not simple-minded. Don't select a simple seeming solution without making sure it solves the problem correctly.
- You won't be the only one to read your code. You need both program and module documentation and in-line comments. Always document the program design, each source file, and each procedure or function within a source file. You will not be the only one to see this code, and those who follow you will appreciate being able to find out what the code is supposed to do and how it is organized. Also, don't forget the in-line comments. Avoid trivial comments that say obvious things about the code. "
t++; /* increment t */" isn't really useful.
- Reveal your Inner-Most Secrets. Ok, you don't actually have to reveal your private thoughts, but if you spend time on choosing between different ways to do things, document why you chose the method you settled on, and if you change how you do things, document why you changed your mind -- what you tried and what didn't work (and even how it failed, if you can figure that out.) This helps prevent the same mistake from being made twice.
- Despite what some Java advocates might tell you, pointers are good; pointers are your friends. Like your friends, you shouldn't hesitate to call on them when the need arises, but don't bother them when it's inappropriate. If you do, they might tire of your constant badgering and become less friendly. (This is a cute way of saying that you don't want so many different pointers in your code that you can't keep them all straight.) More on pointers as this list continues.
- Make complex decisions or calculations as infrequently as practical. If you can design your code so that the decision path in the common case is the shortest, that is best. If you create a function or procedure that makes the decision or calculation, and another that uses the old value of the decision or calculation, and these routines take the same parameters, you can use a function or procedure pointer that you initialize to the slow path and gets set to the fast path when the calculation is made. You should know, in the code, when changes take place that may invalidate the old result, and in those places you set the pointer to point to the slow path again. This is easy in C and easier in C++. This can also be abused. Remember that calling through a pointer is slower than a direct call, but is probably faster than a switch/select, or complicated "
if ... then ... else if .... end-if" control statement, so weigh the savings of the reduction of calculation overhead against the indirect call and book keeping needed. Also, if you do this, document that this is how you're doing things and be consistent.
- Be conventional. If you choose to use a particular coding or naming convention, stick to it. Selecting such conventions for a project (especially a multi-person project) can be a traumatic experience, and for some it verges on religion. Once the decision is made and code starts to come together, it can be very difficult and painful to change conventions.
- It's ok to reshape the world, but reshaping the code is a whole other ball of wax. Especially when using a revision management system, large scale reformatting of the code base makes tracking incremental changes very difficult. If you have to do it, make no other changes to the code other than formatting, then make sure it produces the same object code as before. Binary compares between the object files are not useful here, but in general, the code text segment size should be the same. (If the reformatting changes the number of source lines, you're likely to see object file sizes change due to source information in the symbol or debugging segments in the object file. Every environment is different here.
- They may be small, but there certainly are a lot of them. Keep your source modules small. This way, you run into less chance of two people on a team needing to modify the same file at the same time. A logical group of said small source files should share a common include file that presents a unified interface to the rest of the code and hides the fragmentation of the subsystem source code.
- "If it ain't fixed, don't break it!" Avoid adding new features to a program that's broken; it just adds more suspect code that has to be considered in debugging. Fix the existing bugs, or at least isolate them, before getting too ambitious.
- A novel begins with a few words, an epic with a few novels. Any large software project requires documentation, and more importantly, the right documentation. What follows are a few guidelines for this documentation, but there is no such thing as a complete list of this sort.
- Know what the problem is before you try to solve it. What is the program/package/system supposed to do? You need to look at the problem space carefully. The people who require the project don't always describe it very well. Write a requirements document and make sure your "customers" read it, understand it, and sign off on it. They may supply a list of requirements, but you should also write one, incorporating what you learn from theirs. Yours should include the environment the package will be running in and how it will behave. This will be your target when writing the functional specification, and is the benchmark for how the product meets the requirements when it is deployed.
- Know how you're going to solve the problem. Once you have an agreed upon requirements document, write a functional specification that describes how the package will address the requirements. This document should cover the interaction between the package and users and other programs/packages/systems. These are the external interfaces, not the ones used only internally to the subsystems that make up the package. A functional specification does not outline the internal structure of the package; that's the job of the design, or engineering specification.
- On sound foundations, empires are built. Based on the requirements, as filtered through the functional specification, you need to start out with somewhat detailed design before you start to code the bulk of the application. The design, or engineering, specification is a document that outlines the major components of the package and the interfaces between them. Don't sweat the little details; this is still a fairly high level overview of how the program is to be structured. At this level, you must describe the programming methodology, special tools, and strategies to be employed in creating the program or package. You should be careful in the definition of the inter-module interfaces to avoid painting yourself into a corner, but still give enough direction that an implementation and test plan can be developed from it.
- "Many links make a chain that is only as strong as its weakest link." From the design specification, an implementation plan should be developed. Each subsytem or module described or implied in the design document should be covered, and the order of writing and integration of the modules lined out. There should be a set of milestones with a well defined set of conditions that must be met. This plan includes the integration stages of the project as milestones.
Parallel to the implementation plan is a test plan that outlines the strategy for testing the code base at each milestone, and lay out the limits and boundary conditions that will be tested, along with establishing a procedure for adding failed tests to a regression suite. Test plans set out the minimum for verification of the system -- those things that must be done -- but are non-inclusive; tests that are not included in the test plan are permitted and, in most cases, required.
- "All constants are variable." Just as almost no battle plan survives the first shot of a conflict, no software engineering document remains unchanged during the process that leads from specification to product. The requirements, even though signed off on by the customer, will change. Conditions discovered during the implementation phase will ripple back up to change the design and functional specification. Compromizes will be required. Conflicts will arrise between requirements. Reality will conflict with specifications. Everything you know is wrong, but some things are less wrong than others. Despite all of this, no project implementation plan should end with the heat death of the universe; eventually the requirements and the implementation will become "close enough".
- Socket wrenches are nice, but not useful for removing wood screws. Even if you love a particular set of tools, if they are not appropriate to the task at hand, don't use them. There are some places that the appropriateness of a tool is obvious, other places it's harder to tell. eg: Don't use Perl to do numerical analysis.
- When you get to the finish, be sure you know how you got there. When you build a product for delivery, either to an internal customer or to be put into shrink-wrap packages that will be sold to end-users, you should record which tools are used for the build. Compilers, linkers, libraries, make tools, and any other software tool that is used in building the final project should be noted, including the versions of those tools and components. You want to be able to reproduce the build in the future if you need to build binary patches.
- Just when you think you're finally done, it all starts over again. Unless a non-trivial program/package is retired, there is always another version on the horizon. Sometimes it's an incremental change that can be noted in an addendum to the existing design, sometimes it is a whole-scale rewrite of the entire application, in which the original engineering documents can be used as a basis for a new set of specifications. The long term test plan of the earlier implementation should be incorporated into the new test plan, and another document, the transition plan, is also required.
- Plan for change. When replacing an existing package or program that is part of a business model, a Transition Plan document should be written, detailing the steps that will be taken before, during, and immediately after the transition from the old system to the new system without loss of data or availability in case of an unforseen obsticle that arrises during the transition attempt. Whenever possible, do not destroy the old infrastructure when the new one is put in place, and do several "dry runs" on real data where copies of the existing system are transitioned to the new system and tested. Write scripts for the people doing the steps of the transition to follow. Document everything you do during a test transition -- everything that works, everything that fails. Log your terminal sessions, if applicable. When you are confident that you fully understand how the real system will react to the changes, and the exact script to be followed, including what the person following the script might see in response to commands and actions, then you are ready to make the attempt. It is always good to be able to fall back on the old system should the transition fail, but this is not always possible.
Feel free to reply with additions, refutations, criticism, job offers, or any other comments you feel like making.