Agile

Do agile teams model or write documentation?

To be honest, I had the misconception that agility and modeling are at the opposite poles, that agile teams right little or no documentation. That all changed once I read an article from Dr. Dobb's Journal. Some of the reasons agile teams do up-front modeling is "to answer questions around the scope that they're addressing, the relative cost and schedule, and what their technical strategy is." Another reason is to better grasp and manage the complexity of system architecture.

Some agile modeling an documentation best practices are mentioned as doing "some initial requirements and architecture envisioning early in the project to write executable specifications via a Test-Driven Development (TDD) approach, to single source information whenever possible, to write documentation later in the lifecycle, to promote active stakeholder participation, to implement requirements in priority order, to include modeling in iteration/sprint planning activities, to create models and documents that are just barely good enough for the situation at hand, to model storm the details on a just-in-time (JIT) basis, to sometimes model a bit ahead to explore complex requirements, and to take a multiview approach via multiple models".

One of the complaints that exist with using agile methodologies is that they cannot be applied on large-scale projects and in large development teams. For those kind of projects, plan-driven, model-based solutions are better suited. To achieve this kind of scalability, there is an agile version of Model Driven Development (where MDA is one example of it), called AMDD or Agile Model Driven Development. The difference is that instead of creating extensive models, you create instead agile models. Furthermore, with AMDD, you do just a little of modeling, followed by a lot of coding.

On System Design

In an essay entitled "On System Design", Jim Waldo, a distinguished engineer with Sun Microsystems, expresses his ideas and opinions on system design, including factors that hinder the effort necessary to do system design.

Some of the techniques that aid in the process of good system design are Six Sigma, or UML. The latter helps in the development of good design, but it is you (as an individual or a team) that has to come up with the design. UML helps you in expressing that design better. Many approaches have been found to work, but there is no general approach that works on all projects. Depending on the project, you have to decide which approach better suites your needs. The only exception to this is given by Brooks, namely that good designs have one thing in common, that is they are produced by good designers. Jim Waldo explained this in terms of probability/statistics: those who have been able to produce a good design in the past are far more likely to produce a good design in the future. Furthermore, designers have to be bale to deal with ambiguity while creating the design, and to know that the question that cannot be answered right away, will be eventually answered at the appropriate time. The author mentions that instead of learning a process of design, it is better to learn how to best design. An indicative to the best way to learn how to design includes two factors:

1. Trying, failing, and then trying again.
2. Being under the guidance of someone who is an expert at the task.

It si the way of thinking that matters most in system design, rather than the subject that can be thought in a particular course. Interesting enough, the author mentions that System Design cannot be learned in one course, but should rather be though in a much larger process, similar to an apprenticeship. The relationship needed between a master and its apprentice is found at the graduate level, where each student has an adviser with whom he is involved in academic research. One other place is at work (in industry) where you get hands-on insight on system design. I would argue that at work, you are more constrained in terms of time; hence, there is less opportunity in learning system design.

Diversity in the sense of academic backgrounds (physics, philosophy, anthropology, computer science, music, etc), and how such a diversity creates different views on a given problem, is described as being beneficial. I would particularize this to software development in the sense of not trying to think of a solution for a problem in terms of a particular programming language, but instead design the solution independent of the software platform and only afterward think of what language can help me express this design the best (similar to the MDA concepts).

The problem with doing design up front is the fact that no immediate code is produced. The pressure to produce something that can be seen, or the measure of productivity by means of the number of lines of code written, still exists. What is needed is "an act of faith by management". He continues by saying that "if you find a manager who is actually willing to give you time to do the design task, stick with him or her. He or she is a treasure much rarer than gold."

The paper also mentions intellectual property (as a note on why system design is not shared among peers anymore), and how any patent has the same value as any other patent. For me this is unbelievable; it is equivalent to saying that ideas that are behind the pattern are the same, namely have the same value. The quality of an idea is not taken into consideration at all by the patent system. Furthermore, in case of a patent litigation, the jury is made up of ordinary (non-technical) people, not technical peers. Hence, "it is very hard to learn about good system design unless you can see and study other system designs, both good and bad. The intellectual property atmosphere in industry has limited the number of designs that are actually talked about, and has convinced many system designers that they should not even look at the designs that are available."

Ways of solving the lack of system design from academia and industry are given in forms of agile methodologies and open source projects, because they enhance the discussion of a system's design in several ways: the communication between two (or a small group) of programmers can help in clarifying and better understanding the design of a system; furthermore, communicating the design to other members of an open source project will help clarify the design of a system. It is interesting to note that the author provides a different view to both agile methodologies and open source projects, namely them being a reaction to the impossibility of doing system design in academia and in the industry. Overall, the paper makes for an inspiring reading. You find yourself nodding and agreeing with the statements made while reading the essay.