What this program did was just try to pawn off complexity onto the user. It did not try to solve a specific problem or reduce complexity, it just stated that any program could be created from this program. It didn't say how much work was involved in trying to solve your business problem.

This has lead to some observations on how a project works with and deals with the issue of complexity.

Complexity Exists
By the very nature of software development, programming is a complex profession. There is the complexity of languages (C++ say over Java) where just learning how to design and program a moderately sized application can overwhelm many a programmer. Not to mention the proper usage of headers, template, defines, API signatures (_STDCALL), OOP, Exceptions, pointers, const references and the like.

Complexity also exists in what programmers are asked to do with a language. The creation of even basic GUI applications used to be a very complex task, as the programmer had to work with the raw Window's API and events. Then the MFC class libraries were created and that both added a new level of complexity, but allowed more complex features to be added. Now we have design patterns that provide a method of structuring complex processes while still increasing the sophistication of applications. With the Internet, the added complexity of time, speed, scaling, threading and session management appear as requirements that must be supported.

With each new level of complexity, there are always advancements (ie, changes) that help alleviate or manage these new difficulties, but never reduce the complexity to the same level as before. It's an ever continuing climb to the summit that will never be traversed.

Isolate Complexity
By viewing any application for the web or the desktop, the complexity as a whole can be very overwhelming. But one should never try to design or develop an application with out first isolating the domains of complexity into as small of domains as possible. Even then, each process should be looked at for isolated internal pockets of complexity.

When the decomposition of complexity is complete, then it can more easily be understood, re-designed, rewritten, replaced at any time with minimal impact on other operations. A common problem is to design and build processes that are so complex that maintenance of the system grows unwieldy.

Understanding the complex areas of a project takes time. What at first may be seen as a single large area of complexity may in fact contain multiple domains that can or should be separated. A good example of this is the area of user interfaces. In the early days of Windows programming, the graphical parts of a program accessed the processing areas of a program and visa-versa. Very little though was given to limitations and affects of having some data processing code access a UI control. That's just how things worked (even with many VB programs today) in those early days.

From this frame of reference, the GUI application was a large complex process. But if we take this GUI process and review it further we can break it down into multiple areas of complexity that could lead to greater flexibility and maintainability.

For example: The GUI of an application may appear to contain a single large complex set of operations, really is comprised of multiple complex domains that consists of :

• Menus and Commands (Controller)

• Visual views of data (views)
• Data objects (Model)

These multiple domains would match what is called MVC (Model View Controller) pattern. This pattern is nothing new. It was first created with the development of the SmallTalk system in the 70's. It just took a while for the rest of the world to catch up.

By looking in the Menus and Commands Domain, there are further areas that can be isolated. The Menus and Commands domain can be broken down even further into:

• Command Pattern

• Undo Manager Pattern
• State Manager Pattern

From this example, the programmer must first understand the possible complex domains and then research the possible solutions for each.

What is easier is just to add code to enable/disable control where ever states change. This works up to a certain point. As features are added, changed and removed, there will be states that will be missed and the resulting errors. No problem you say, we'll just fix them as we find them and if we need to rewrite it later, we will. Sounds like a good plan so far.

Complexity does not always mean compact, tight code that only the lead programmer can understand. Complexity can also appear in the form of an applications wiring that is intertwined through out processes and almost goes unnoticed. This is more subtle form of complexity because as it grows, it effects most and more of code until unwanted side effects start appearing when maintenance is occurring. Looking at the Menus and Commands domain, we can see this type of complexity as they may be interspersed through out the applications frames, windows and views. The logic for enabling/disabling, processing and undoing operations may be scattered thought the code.

This type of complexity is the reason for the existence of the MVC, Command pattern, the Undo/Redo pattern and the State Monitor pattern among others. There are many other patterns that reduce complexity at the expense of a bit more understanding and restructuring of code, but are well worth the extra time and effort.

Managing Complexity
By being aware of complexity in a projects design and implementation, you can focus on managing these area. Managing complexity involves tracking the growth of complex domains, looking for the formation of new complex domains, providing solutions for the reduction of complexity and researching the repercussions of coding solution in the context of complexity.

Managing complexity is not the same as moving it around the code. This is a common mistake. Programmers can reduce the complexity of a product, if the user has more to setup and configure. The program "The Last One" made this mistake. It did not reduce any complexity of the product, but only shifted responsibility onto the user, to design and implement logic operations before the product would be of any benefit. That's a lot to ask of a user. This was not much different than selling a language IDE and promising that you can code any solution.

Shifting the responsibility of complexity is a common development strategy, but like a child that hates eating peas, no matter how you move them around on the plate, they're still there.

Buying a Solution
Ah the famous idea that purchasing a solution will reduce the complexity. The kitchen recipe card catalog is a good example. Early on in the computer revolution, uses for a home computer included having your cooking recipes on a computer for fast access. You can still find these programs in the discount bin at stores. The idea is that by having all of you receipts on the computer, a person could search for all recipes with chocolate or scale up a measurements for servings of 18 instead of 12. The goal was to reduce the complexity of working with recipes by the purchase of a computer. The problem is that interfacing with the computer was worse than hunting through the stack of recipe cards. Yes, the computer could offer features that the card stack could not, but getting to that point proved to be too high of a hurtle for most people.

This analogue is the same for programmers. Everything has it's price and using purchased components also comes with a price. To determine the price of buying a component, you may want to ask yourself the following questions:

• What percent of the solution is solved by this component?

• Is this component designed for our solution, or are we trying to shoehorn it into our solution?

• Is the complexity of interfacing more complex than the complexity we are trying to solve?

• Are there any other restrictions the component places upon it's usage (file formats, concurrent usage, source code / binary only, memory / speed issues, licensing / price, conflicts after deployment / support) that would impact the project?

These issues should all considered when evaluating using components to solve your complexity issues. Many VB projects have ballooned in size because purchased components had dependancies on still other components. What started out as a simple 500K product, now requires 20MB of supplemental code, code that requires testing, installation testing and support.

No, there is nothing wrong with using purchased components when it's understood the reasons for doing so. When used for the right purpose this can speed development and provide a reliable product.

Now that buying a component sounds scary, the flip side can be just as scary: Writing the code instead of buying a solution or build vs buy. Because components for sale have to appeal to the largest market as possible, they may contain more features than are required for the project, or have too many requirements of their own, licensing issues or any number of issues that just make this solution not work.

Reasons that should be red flagged for building components vs buying them are: the initial cost and the programmers ego's (we can do it, faster, better, cheaper). The cost issue is really a red herring as the cost of developers time will almost always be more than any purchase price. The ego issue is a little harder to spot as it can be disguised in a multitude of justifications, some valid, many ego based. Remember that programmers love to code and almost any reason will be given to justify this personal goal.

Common reasons for the "build our own" path of development are:

• Our needs are special and different from others.

• We are much better programmers and can provide a better tool.

• We can do it much cheaper and faster.

Looking at these stated reasons, you can see that they could be true, but could also just be facades for wanting to write more code, so the possible reasons need to be explored in depth to determine their true validity.

Special Needs
For the special needs reason, it needs to be asked, why are you so special? Are your needs so different from all of the other software firms that you need to spend valuable time and resources to develop components and tools that only this project will use? That the project will be a failure without this extra internal development? By wording the questions in this manner, the onus is placed on the programmers to justify this special requirement. Ideally this unique requirement should have been identified during the research and design phase of the project, but needs may have changed over the course of the project.

DownStream
There exists a common tactic that is used to sway technical one way or the other. When the requirements are defined for a new tool, language, component, or development solution, some "special" requirements are injected into the specifications that are designed to sway the decision a specific way. If only a single special interest uses this tactic, then the odds are increased that the decision matches that of the special interest. If there are multiple special interest, then what can happen is that a deadlock is created because no single decision encompasses all of the special requirements.

Example
A middle ware messaging system was being developed by a company that used multiple servers in their web application. Cost was a special requirement (meaning free) even thought the costs of purchasing a component was not that high. The decision was to implement the messaging using Sockets. The programmer that implemented the solution did not use basic sockets, but instead used a propriety API for the sake of performance. Now, performance was never on the list of requirements because the messaging demand on the servers was known to be low, but this special requirement was imposed by this programmer to justify the deviation of implementation.

The messaging code was development and was buggy and no one, not even the programmer knew how to fix it. A meeting was called to decide which component to purchase to replace the buggy in-house code. This new requirement of high performance, using this special API was now injected into the decision process. This caused a deadlock as this programmer demanded than any external solution, use this special API.

The dead lock was only broken when outsiders to the team, were brought in to evaluate the requirements. The special API was dropped, the high performance was dropped and a new programmer was selected to implement the solution. This new implementation was completed very quickly and the project moved on.

We know whats best
This reason is just a variation of the Special Needs reason except that instead of a build vs buy decision, this is more side slipping in of code with no formal justification or review. You can see this in the subtle form of a team creating their own collection classes, to writing there own UI wrappers around a perfectly good set of UI classes. This is the "We Know What's Best" perspective. The justification is that they can provide a "better" set of classes for the projects (or companies) programmers, than the language vendor can. Of course the increase in development efficiency is suppose to out weight the design, development, testing and support costs of these "wrapper" classes, even though they are based on the built in classes and may never be used outside of this project. A good cost estimate of the time and resource impact of this approach, can do much to separate a real need from an ego based desire.

Cheaper and faster
This justification will fly in shops that have limited defined processes and requirements. If the normal requirements of design documents, code review, and unit tests are factored in, the cheaper and faster argument starts to loose it's appeal. But without questioning what the definition of what "cheaper" and "faster" really means in terms of a complete (tested, documented, etc.) component, this choice sounds appealing. The "I can get it done in one week" sounds like a real time saver. But marketing is about moving a product, not about delivering what is needed.

Example
A communication component is required for a web application project. The component cost about 3,000 dollars and would require about 1-2 weeks of integration time.

Reasons given for building a component:

• We can do it in 2-3 weeks.

• It can be more flexible.

• We have complete control.

• It's free.

Reasons given for buying a component:

• It's already developed.

The main points were brought up in a meeting of building the component verses buying one. The underlying focus in the meeting was one of finding the cheapest route. Based on this underlying force, the decision was to build the component in house. The first result was that the component was built and worked within the narrow requirements. The additional programmer time for the construction was roughly equal to the purchase code of the component, but there was a larger secondary effect.

This secondary effect was that the application started to have random failures that could not be identified. Each section of code was looked at and no point of failure could be found, but the errors continued. After a number of weeks spent in extra testing and debugging, it was determined that the home built communication component was at fault. The programmer that created the code had unit tested it and could not find any faults, yet it failed in the integration tests. The source of the error was that the component would work fine if the the packets were exactly as expected, but if not, then the component would overrun memory and at some point later, cause a GPF. The unit testing was all done with valid packets, it was never tested to fail!

The extra development and testing time now far exceeded the original goal of reducing costs. Just as in the unit testing, the original points for building the control only stated what could go right and not what could go wrong.

Reducing Complexity
It was Einstein who said "Make it as simple as possible, but no simpler". True in physics as it is in software development. The reduction in complexity can come in the form of simply reworking code, changing an algorithm or using a little inheritance there. These are the easy methods of reducing complexity, and do not require any change in mindset or re-evaluating of a design, just a straight forward reworking of the code.

These simple coding methods can only manage complexity so far. What if the problem domain is by nature complex and your basic designs only compound the complexity? When a project reaches a certain size, there may be areas that just can not be scaled up using the same coding techniques and designs. Then it's time to search for different types of solutions, and these solutions exists in the form of design patterns.

The difficulty is that programmers like to find the quickest way from point A to B, and understanding and researching design patterns is counter to that thinking. Programmers are more comfortable with code than designs. At first glance the notion of implementing a design pattern may seem overkill for the current problem. That by adding code, the resulting product will be even more complex than before. But just the opposite occurs. Yes there is a learning curve, but design patterns allow the developer to reach a new level of development that is other wise unattainable to them.

Patterns allow the organization and viewing of objects and processes and their interaction. This allows placing much of a program's complexity into a separate patterns structure and removing it from from the original objects.

The benefit is the reduced complexity in the coded objects, but at a cost of a defined structure imposed by the pattern on those coded objects. It's this imposed structure that goes against the quickest point A to B mindset that is so prevalent in development. Digging deeper, this is also similar to the resistance to imposing structure on software development in general. Programmers that argue against development and process standards, are also likely to resist employing design patterns in their code.

Example
Some features can not really be added in afterwards in a later release. One such feature is that of unlimited undo/redo operations. If an application is not designed for this feature, then it's almost impossible to include it in a later release with out extensive re-working. In this case going from point A to B would require some additional learning about the Command pattern, something that would take time and effort on the part of the programmers. If a programmer did not understand the Undo or Command pattern then it may be assumed that it could be added in a later release or later in the development process. But with the research of the feature before hand, an understanding of the pattern's operations would enable programmers to adequately design and code for it's inclusion.

Limiting Complexity
Limiting complexity is not adding complexity that isn't needed. Sounds simple, but it happens. It manifests itself as features or requirements that are not useful or required for the user of the product. An example of this is an elaborate customized configuration for the application (we're talking desktop here, not web). This may add very little to the usability of the product but requires many resources to design, develop and support. Another example is that of over use of graphics or Flash on web pages. They look nice, but may inhibit the usability and web experience (distractions and delays) of the user. Yes, image maps and animation may look "cool" but should the user be exposed to this every time the page is viewed?

Example
A product is designed and coding has started. The design calls for configuration files that contain the reporting formats for all output. These configuration files are only used internally, for designing the reports. Thus it was also requested that a report design application be created to facilitate the creation of these files. Hum, the projects resources have been greatly expanded by the development of the reporting tool. Would XML and XSLT have worked better for the reporting configuration files? Would another technology work equally as well, but without the additional resource consumption of an application? Any development addition to a project should have direct benefits to either development or to the user.

Downstream:
The company (and you) get paid for a successful product. When unneeded code is added (or requested) to a projects, as defined by the projects goal and vision, then it's like reducing the profitability of the product. This reduction in profitability effects everyone involved with the project. If the unneeded code is too complex or require too many resources, then the project may fail all together, with dire results for all.