Something interesting about designing a game is that you can start in one of two ways:
1. Write down some rules.
2. Put together some chits/models/boards/etc and play with them.
In the first case, what you usually get is a mess of ideas with no actual relation to how people might actually play together unless you have an incredible imagination, and write rules like Alan Moore writes comics. There are plenty of sets of rules out there where the author has not taken the time to play the game, and it shows in a cursory reading of the rules. Some plans do not even manage to look good on paper.
In the second case, what you usually get is something that makes sense for the game between you and whoever you were playing with, and woe betide you if it was solo-play. Codifying the rules governing that play, and extending them to elements that were not a part of the original design-play, will take what you were doing and carefully exsanguinate that activity so it can be stapled to a page to mystify end-users that are not you.
Designing a game requires both, the skill to conceptualize the architecture of the game, and the skill to isolate the hook, the feature of that experience you want to share as a benefit with other players. A good process, insofar as processes go, for game design involves finding a reflective equilibrium between design and alpha-testing. You play, and then write, and then play again. However, contrary to certain opinions (and why I parted ways with the M42 Project) the writing part is really important.
Writing rules down well, and according to the best practices of technical writing, is important because whatever the rules represent, they need to be clearly communicated to whomever is testing them, and to the end-user. Sets of rules are simply collections of processes and procedures, and need to be communicated as such. Nonetheless, some people, particularly authors who are writing rules as an esoteric corner of the gaming hobby, believe that sets of rules are ends in themselves, and so need to be entertaining. The excuse is that people will be bored reading rules that are expressed in a clear and usable format; that this information, or 'rules-text', needs to be interspersed with flavour-text telling the user how the rules represent the game's narrative.
Take the Flames of War rules, for example. On one hand they are clearly written by table-top game standards, and on the other they still blaspheme against the standards of the Society of Technical Communicators. Flames of War attempts to separate the text conveying the rules with the 'flavour' text informing the players about what the rule is intended to represent. However, the writers manage to distinguish the rules-text, the content, from the flavour-text, the decoration making a set of processes and operations resemble a World War II battle in the minds of its players, by italicizing the rules-text. All of it. And that's on a dark, low-contrast background.
Something that game designers forget, perhaps because they work in the entertainment industry, is that rule-documents, including rulebooks, business unit process documents, manuals, and so on, are not the end-product of their work. The end-product is the game, which the rule-books support by telling players how to play the game properly. Once players have internalized the game, there is very little reason to read the rulebook again, unless there is an argument regarding differing interpretations of the rules.
That's one of the secrets to writing good technical documentation: realizing that nobody wants to read what you have written. People come to manuals and other documentation grudgingly, at the end of their ropes, when they are tired, angry, and annoyed with the product that they cannot figure out on their own. Good documentation minimizes the barrier between the user, and the information that they want to find and use in pursuit of a different task than reading a document.
Going back to game design, the designer needs to understand that while their task is to create a game, rather than write a set of rules, they need to be able to write those rules so that they support users playing that game. In a general sense, good documentation practice runs parallel to the main flow of production, both a concurrent product, and a tool to support the production as well as the main product.
Documents that should be created before the design process initiates include:
Game design document (game specifications, hardware as well as rules, purpose, content, etc)
Project plan (setting a schedule for the design, if you ever want the design to be an actual game)
Playtest feedback form (different for alpha and beta testing)
Usability feedback form (games need to be usable as well as fun)
Game workflow diagram (a top-level view of the game from beginning to end)
Work tracking sheet (used for tracking work through the schedule, and co-ordinating production)
Project version tracking sheet (in the absence of decent versioning software, like subversion)
Project framework (detailing processes for editing, testing, and other production activities).
Long story short, writing rules in a technical fashion may take some of the fun out of writing up rules, but it goes a long way to supporting both the design and user in sharing the experience of a good game. Rather than disguising a good game with incoherent rules, or preventing a good game from seeing production through disorganization, designers should consider that they are producing game for other people, and well-organized communication is how they create that game and reach those people.