To simplify things, the company in question is simply CompanyX and their main product is ProductY.  This document was originally a Word doc which I had to paste into Notepad and then into WordPress and then manually edit to restore formatting.  So please forgive any formatting mistakes as it was a lengthy manual process.

Overview

As requested, I have assembled a body of knowledge and best practices in process management with which our organization may promote healthy, productive software development and software development management.  Because CompanyX focuses solely on software as a service (i.e. “on demand”) products, many traditional approaches to process and project management do not apply or must be applied with an eye towards modifying their principles to work most aptly within CompanyX’s corporate framework.  This document is an attempt to distill the general body of knowledge available in this area and to modify it so that it is applicable and usable within CompanyX.

As you know, CompanyX’s approach to software development has always focused around forward-thinking, progressive hiring practices which aggressively seek to hire the absolute best IT professionals and extremely agile development practices designed to leverage the skill base of our developers most effectively while reducing overhead created in other organizations necessary to manage mediocrity in the environment.  Because of these practices it is necessary for any software development discussion within the CompanyX context to take these into consideration and to therefore alter generally accepted industry norms and practices to reflect this.

The basis of this document is not to enact change unnecessarily but to identify potential areas for improvement, to find areas which are weak, missing or poor and to codify practices which are working and should not be changed.  This document is very timely as CompanyX is currently undergoing a change from a single-focus medical facilities software firm to a multi-focused firm with a financial software group and internally managed hedge fund.

Current Development Status

To begin our discussion, I would like to look at the current state of the CompanyX development environment to establish a baseline against which we may compare our proposal of best practices and design.
CompanyX has two key datacenters, Chicago (internal and development) and Houston (production and Internet facing / non-VPN applications.)  Houston is in the process of being relocated to San Francisco.  A third datacenter, dedicated to backup storage and warm recovery needs, is being planned in Philadelphia.

At this time, CompanyX employs only a handful of developers most of whom do not need to directly collaborate as each works in a highly segregated technology niche which creates a situation of relative exclusion simply by its nature.  All CompanyX developers work from home offices provided by CompanyX using CompanyX workstations, software, networking hardware and telephones (when appropriate.)

Collaboration is handled through the secured corporate XMPP and eMail servers as well as over VoIP.  Remote desktops are visible using the central NX remote desktop acceleration system giving all employees access to one another on the central network.  Centralized documentation is handled through the corporate wiki.  Source control is handled via Subversion hosted at Peoria Heights – closest to the development servers.
Each home office has a hardware firewall with IPSec VPN acceleration to make networking to CompanyX’s Peoria Heights office transparent and simple.  Files can be transferred directly via CIFS or NFS from the central filers.

The main development platform at this time is Visual Studio 2008 using C# 3.0 and ASP.NET 3.5.  Visual Basic and VB.NET has been eliminated after years of weaning.  SQL Server 2005 and MySQL 4/5 are the key database platforms.  PHP is in use for some internal applications.

Given CompanyX’s current small scale this environment has proven to be quite effective, but we must realize that as growth and diversification occurs it will become increasingly necessary to add more structure and communication methods in order to keep projects moving fluidly.

Physical Location Issues

The first issue in development environments that we will address is physical space and location issues for the development teams.  Conventional wisdom says that the most practical means of getting maximum performance from developers is to locate them physically near to one another to facilitate rapid and smooth communications.  The need for rapid communications, however, is in direct conflict with the need that developers have for quiet and uninterrupted time for development to allow them to efficiently enter and maintain “the zone”.

“…knowledge workers work best by getting into “flow”, also known as being “in the zone”, where they are fully concentrated on their work and fully tuned out of their environment. They lose track of time and produce great stuff through absolute concentration. This is when they get all of their productive work done.” – Joel Spolsky, Joel on Software

In weighing the benefits of high level communications versus the benefits of isolation, flexible time, flexible location, travel, etc. we have determined that the cons of collocated development teams outweigh the benefits in our case.  Our approach, at this time, is to continue to pursue a completely work from home environment encouraging extremely flexible schedules allowing developers to maintain “the zone” for longer than usual durations while encouraging extended downtime when “the zone” is not obtainable.

Physical Environment

Regardless of the physical juxtaposition of developers a superior development environment is a necessity.  Following the recommendations of Joel Spolsky, a recognized expect in physical development environments, we intend to make Herman-Miller Aeron chairs available for all developers along with two 20.1” LCD monitors as a minimal, standard setup.  Additional monitors and choice of development platforms (Windows, Mac OSX, Linux, Solaris) are available at the request of the developers.  Also, all developers have access to commercial grade Bush desks that can be shipped directly from or picked up from Office Max.  These desks have surfaces that are very conducive to heavy mouse usage.

Communication Tools

Given that intra-team communications are of the utmost importance but recognizing that we are unable to effectively co-locate most of our staff we must turn to technology to mitigate this problem as far as is possible.  This is a challenge faced by all teams even when they have the luxury of physical proximity.  We believe that we hold an unapparent advantage in this area because we are acknowledging communications as a shortcoming up front and tackling this issue seriously rather than assuming that communications is handled vis-à-vis natural proximity.

We have a number of tools at our disposal that should function well in the role of providing a solid communications infrastructure for CompanyX.  We have many of these tools in place already such as a very successful email system and groupware package that handles basic communications both internally to the company and externally to clients and vendors.  The system is secure, reliable, readily accessible both inside and outside of the company, is tested to work well with BlackBerry and iPhone devices, etc.  We have team aliases set up to encourage team collaboration in this form.

Our email system also includes a simple document management system which has not proven useful for team collaboration but has shown to be very effective for individual user note-taking and reference information rather than storing information directly in an email format but still allowing this data to be accessed easily through the convenience of the email interface.

Group calendaring is currently proving to be a very useful addition to the eMail and groupware system.  Calendars are easily viewed by interested parties and teams can select calendar views to allow them to see the available and busy times of everyone on their team all at once in a single calendar view which is very effective for ad hoc planning and collaboration events.

In addition to asynchronous, archival communications available via eMail, CompanyX focuses heavily on the use of the XMPP/Jabber protocol for instant messaging via the OpenFire Server product.  This tool has worked very effectively for us in the past and our plans for the future involve a continued investment in this technology and an advancement in our client-side architecture with which to support communications. Currently we have a Java-based client, Spark by Ignite RealTime, that serves very well for general use.  We have begun testing an Adobe Flash based client that is available via a web page that may prove to be very useful for staff who need casual instant messaging access from machines where Spark is not installed.  It is our goal to make secure instant messaging access simple and ubiquitous so that staff are available to one another, in real time, whenever possible.

Often staff is required to be live on the instant messaging infrastructure while located in a secured office location that does not allow direct XMPP/S communications to the Internet.  For these users we have implemented an SSL Proxy / VPN product, SSL-Explorer, which allows them to access our applications such as eMail and XMPP even under the harshest network conditions.

Recently, CompanyX has moved from pmWiki to DokuWiki and has begun focusing heavily on using the wiki format to support as much internal development documentation as possible.  The wiki application is ideal for much of development communication because it is easy for any collaborator to create necessary documents, all contributors can edit their appropriate documents and staff requiring information have a central, searchable repository in which to perform searches for necessary information.  Wiki documents are automatically versioned making it trivial to track changes over time and to return to previous versions when necessary.

The wiki format offers many benefits that would be considered non-obvious.  DokuWiki allows staff to subscribe to wiki pages in which they have an interest getting automatic change notifications, either via eMail or via RSS/Atom feeds.  Through this subscription system the people who are responsible for or dependent upon a particular set of data can find out about changes immediately and react accordingly even if the person modifying the document is unaware of which parties may be affected.

The wiki format is useful for many types of documentation with standards documents, best practices, guidelines, project initiation documents such as the project charter and use cases being among the most effective.  The wiki is most useful as a supporting architecture for use cases which are the bread and butter of our design documents at CompanyX.  Their text basis and ongoing modifications lend them to be ideal candidates for wiki-ization.

Not all document types lend themselves well to the wiki architecture.  Rarely changing documents, end-user printable documentation and UML diagrams, for example, are more appropriately handled through a richer format such as Microsoft Word, Microsoft Visio, Umbrello and OpenOffice.org Writer.  To support these file formats a more traditional content management system is recommended.  The two products that are leading, at this time, in that area are Microsoft’s SharePoint Server and the Alfresco Content Management System. Both of these products often portal functionality so that they may function as a central starting point for staff when searching for content as well as document repositories with interfaces to traditional file stores.  Both may additionally be used as supporting frameworks for module application interfaces.

Voice communications remain very important, especially for detailed technical discussions in real time.  In order to support fluid and cost effective voice communications CompanyX will implement an internally hosted Voice over Internet Protocal (VoIP) system.  VoIP compares favorably to traditional voice technologies with many advanced features available such as conference calling, call forwarding, follow me service, voice-to-text support, auto-paging, voice to email interfaces, etc.  The cost of VoIP is also very low compared to traditional voice technologies.  Because VoIP systems are so cost effective, CompanyX has the opportunity to extend the internal corporate voice infrastructure into the homes of all staff providing a seamless environment so that neither internal staff nor external clients can easily tell when a given member of staff are located at home, on the road or in the office.  This flexibility and transparency is important in supporting the disparate office environment required by CompanyX.

Unified Communications platforms, such as those made popular by Microsoft, Cisco, Nortel and Avaya, have many advantages over our recommended approach of maintaining and moving forward with separate eMail, IM, VoIP and other communication products.  By choosing to work with separate products CompanyX has the benefit of choosing “best of breed” products rather than a single, unified product that does not necessarily need to compete head to head with industry leading products.  More importantly, however, Unified Communications or UC fall under a legal grey area, at this time, and may subject loosely regulated communications such as voice to long term scrutiny and archival penalties of communications such as eMail.  Because of laws within our primary jurisdiction, the United States, it is important that UC be avoided until legal hassles surrounding its implementation are resolved conclusively.

Development Methodologies

Development Methodologies are critical underpinnings of software development.  CompanyX has always focused on highly iterative and extremely agile development practices focusing heavily on those methods and practices that lend themselves first and foremost towards heavy innovation, rapid adaptation to the market and quick feature development.

Because of our needs and strategies, CompanyX has turned to highly modified Agile methods drawing inspiration from popular Agile methodologies, especially eXtreme Programming (XP) and SCRUM.
CompanyX has adopted the XP use of the “user story” as the primary measurement against which development is performed.  Stories are not completely “user” related and can sometimes be “developer” stories to allow simple scheduling for changes not visible to the end user.  This can be misleading to XP adherents as stories are designed to be visible to the client but CompanyX’s true end client is, in a fashion, the development organization itself and so we treat stories in an unusual way at times.

In order to support the continuous development processes within CompanyX any standard methodology must be adapted heavily as all major methodologies rely upon release cycles, sprints, etc.  CompanyX uses Continuous Sustainable Velocity as its model and does not have scheduled release cycles.

A development team “burn down chart” or backlog system is used to schedule upcoming stories so that priorities are maintained even without a set schedule.  The backlog is continuously managed by the development lead and the analyst lead / project manager to keep the stories organized by priority but with continuous reference to their weight (estimated time for completion.)  The weight may be used to determine that a small feature or minor change might need to be popped to the top of the backlog stack because it represents better story velocity.

The goal within CompanyX is to get new features designed, developed, tested and into the hands of our customers as quickly as possible.  Because we release all of our software directly to our own servers and deploy internally making the software available as a service to our customers we focus on releasing as features are made available and do not need to announce most new features or changes.  We try to roll out changes gradually on a regular basis so that customers always have some new fix, feature or performance enhancement.  To our customers our system is very much alive – a system constantly under change and development.

Extreme Programming

CompanyX’s methodology also follows the XP principle of simplicity and “You Aren’t Gonna Need It” or YAGNI.  This principle, which states “Always implement things when you actually need them, never when you just foresee that you need them.”, is important to allow CompanyX to produce software more rapidly while delaying potentially complicated and unneeded work until we are sure that it is required.

CompanyX loosely follows Mehdi Mirakhorli’s Rules of Engagement and Rules of Play for XP which I will list and explain:

• Business People and Developers Work Jointly: CompanyX follows this very closely but, because CompanyX does not have direct customers but is itself the primary customer of the software products which it then hosts as a service, CompanyX produces an industry trained business analyst who represents CompanyX as an entity as well as theoretical business customers.  This analyst is available full time as a “business person” with whom the developers can go for clarification, expansion, etc.

• The Highest Priority is Customer Satisfaction: This means primarily that CompanyX is able to host and manage the application and second that CompanyX’s SaaS customers are thrilled with the quality, features and innovation of our services.

• Deliver Working Software Frequenty: In XP, short term timeboxing is used to encourage the delivery of new features and updates on a regular basis.  At CompanyX we attempt to roll features and fixes out as quickly as they become available on a continuous basis.  Delays are only introduced to this system when multiple changes may be available extremely close to one another or when system changes are considered risky.
• Working Software: Our goal is to produce working software.  No other deliverable has the priority of working software.

• Global Awareness: The development team must tune its behavior in order to support the overall needs of the organization and its customers.

• The Team Must Act as an Effective Social Network: Honest communication leading to continuous learning and an emphasis on person-to-person interaction, rather than documentation, empowerment for developers and analysts to do the work that they need to do and to make the decisions necessary to create and innovate and to level the political playing field so that all team members are available to one another and seen as peers.

• Continuous Testing: All code is continuously tested through all reasonable means.

• Clearness and Quality of Code: Code should be clean and elegant resisting duplication and obfuscation.  When possible, code should adhere to the precepts of Software as Literature.

• Common Vocabulary: The team, as well as the organization, should share a common vernacular to reduce miscommunication and increase the speed of development.

• Everybody Has the Authority: Everyone on the development team has the authority to do any work, to fix any problem and two or more people are always available who understand any given piece of the system.

In addition to the “Rules” of XP which CompanyX follows almost exactly, there are twelve XP Practices, as set down by Kent Beck, which we follow less closely.  The practices which we do not believe are beneficial to us include pair programming and test driven development (at all times.)  We feel that these practices are too heavy for CompanyX’s needs.

The XP Practices which CompanyX should be adhering to include: the Planning Game (for CompanyX this is played between the development and analyst leads), Whole Team (business is always available for the developers), continuous integration, refactoring, small releases (CompanyX uses continuous releases), coding standards, collective code ownership, simple design, system metaphor (business and development joint metaphor for system functionality creating a meaningful, shared namespace) and sustainable pace (CompanyX should never approve pushed overtime.)

SCRUM

CompanyX also takes concepts from the SCRUM development methodology but not as heavily as from XP.  SCRUM’s focus is more upon agile software management then upon software development itself as is the focus of XP.

SCRUM separates stakeholders into two key groups, pigs who are completely invested in the project and chickens who have an interest in the project but are not necessarily totally committed to it.  CompanyX follows this tradition but without the humorous naming conventions.

Within the pig group of stakeholders, SCRUM recognizes three key roles: product owner, ScrumMaster and the Team.  The product owner is the voice of the customer and makes sure that the team is working on features and in a direction most beneficial to the business.  The ScrumMaster, sometimes called a Facilitator, whose role overtakes the traditional role of the Project Manager but, instead of leading the team or “managing” them the ScrumMaster’s job is to interface with the organizational management and to make sure that outside influences do not get in the way of the team’s ability to work.  Finally, the team itself includes the developers, designers and others who perform the “real work.”

CompanyX organizes its teams similarly.  The Product Owner role is played by the business analyst assigned to the project who represents the clients or potential clients.  Since SCRUM is generally organized around contract software development their role is more aligned with a contract customer while CompanyX’s customers are potential industry customers and so the business analyst’s role is to have a firm grasp on the business of the industry as a whole and to represent them in a way that no one person within the industry may be able to do so themselves.  These roles are extremely similar.

The ScrumMaster role is less clearly defined within CompanyX.  As CompanyX does not actually use SCRUM we do not require someone to manage the SCRUM process.  A manager is selected to act in the Facilitator role, but we use the term Project Manager, whose role it is to keep the team able to work.  The Project Manager is not over the team and does not manage the team but manages the project, interfacing with the sponsors, providing data, keeping the team able to keep working by not being bothered by the overhead of politics.  Often this role does not require a full time person and this role is either played by an executive sponsor or by the business analyst.

CompanyX’s concept of a team aligns very well with the SCRUM idea of the team being a self-organizing entity that needs no external management.  Like SCRUM, CompanyX keeps teams small so intra-team communication is fast and fluid.  Additional overhead of formal meetings or project management processes are not necessary.

Much of CompanyX’s concepts of project task organization come from SCRUM.  CompanyX’s teams utilize the product backlog and burndown chart systems for providing straightforward and simple project planning, scheduling, etc.  As we do not use Sprints themselves the idea of the Sprint Backlog is missing and, as we see it, unnecessary.  CompanyX does not require schedule detail of the level that is generally provided by the Sprint backlog and so this is seen as superfluous overhead that should remain out of the picture.

Unlike both XP and SCRUM, CompanyX does not generally encourage customers to become a part of the development process.  Ideas and suggestions are solicited from customers at times but these are used to provide vision and guidance but almost never day to day project goals or scheduling.  We choose this model because CompanyX delivers a software product that is then purchased by unknown customers.  It is the job of the business analyst to understand the needs of the potential customer pool and to anticipate their needs better than they would be able to do.

After eight years of developing our methodologies based upon our studies of the software management and development field and upon our unique software development models and delivery mechanisms we feel that our blend of light management, informal processes and extremely agile methodology is quite effective and does not constitute unnecessary overhead while allowing the team members the ability to communicate and work effectively together.

Core Technology Competencies

Due to CompanyX’s relatively small size it is important that we focus upon a core group of technology and platform competency areas in which we are able to specialize and develop a mature practice in which we are able to cross-train and mentor.  The core areas in which we are currently focused include Microsoft’s C# on the .NET platform, Java with Spring and Hibernate on UNIX (both Red Hat Linux and Sun Solaris) and Ruby on Rails.  C# and .NET have long been our core platform replacing our previous VBScript/ASP platform based on Microsoft’s venerable DNA model.

C# and Java remain our most critical platforms providing the basis for much of the code which we currently have in production as well as much of the code that currently exists in our pipeline.  These two languages and library collections are also, by far, the most well known and well understood platforms by our current development teams who have used these technologies extensively both within CompanyX and without.

In the past year, Ruby and the Rails web application framework have begun to appear as an important additional platform choice for CompanyX.  Ruby on Rails is targeted at rapid web application development and often aligns well with highly agile methods like those used at CompanyX.  Ruby is well suited to the average database intensive but computational non-intensive applications which we are apt to produce.  Ruby’s high level of abstraction and powerful library system is highly advantageous for building complex business applications very quickly.

For financial applications and other high computational performance needs situations the use of C or, less likely, C++ may be critical for small pieces of code requiring extensive tuning.  This may not be a correct assumption, however, has the Java platform is extremely performant, especially in numeric computations, and may prove to be more than adequate.  Having a C compiler tested and approved would, most likely, be important to ensure that necessary tools are ready before they are needed.  Our financial development lead is currently evaluating two compiler packages in order to find an appropriate choice for our platforms.  As we are standardized on AMD Opteron and Sun UltraSparc processors only the GCC (AMD Optimized) and the Sun C Compiler (Sparc Optimized) are being considered since both outperform the rather expensive Intel C Compiler for our chosen hardware platforms.

Future Technologies

In the future, having access to a readily available C platform may be critical as CompanyX investigates the NVidia GPU-based CUDA technologies for massively parallel floating point computations.  The CUDA libraries and drivers necessary to access the NVidia Tesla GPU hardware are currently callable only through C which may necessitate the use of this language.

Under investigation, potentially as production development platforms but more often as learning and exploration opportunities, are a number of languages and platforms.  We believe that it is important to constantly seek out new languages, approaches, platforms and techniques so that we are prepared when new challenges arrive.  Experimenting in new languages and platforms is beneficial outside of potential production usage as well.  Having an opportunity to work in a diverse environment is more exciting for developers and also contributes to growth.

Currently we are experimenting with the use of Clojure (a Scheme-like LISP language that runs on top of the Java Run Time), Groovy (a Ruby-like language that shares syntax with Java and runs on the Java Run Time) and Scheme (a LISP variant.)  Highly potential other languages for our use include Scala, Boo, J# and Haskell.  By working with new and groundbreaking languages we expand the way in which our developers think helping them to approach problems from a unique perspective and to discover new and different solutions.

Design Tools

Standardized design tools are important in providing consistent communications and documentation from the design process to the development process.  CompanyX chooses to focus on a limited set of highly effective design tools that deliver us the maximum of communicative power without unnecessary overhead and the creation of design documents that are unlikely to be used and may rapidly become outdated and ineffectual.

We have chosen to use the wiki format as our preferred mode of design document sharing.  Wikis are easy to implement and easy for all team members to access and modify.  Collaboration happens fluidly and naturally on a wiki and built in change control mechanisms protect against data loss even when major changes to documentation are made.  In the past we have used pmWiki but found the system to be lacking in many respects.  After some searching we have, in the past year, migrated to DokuWiki which is much better suited to corporate-style documentation needs and authentication options that allow us to integrate it into our environment effectively through LDAP integration which allows it to authenticate either to our UNIX systems or to Active Directory and Windows.

The wiki format has proven to be very effective from its initial usage at CompanyX and we have begun to use it in many areas including new hire documentation, helpdesk information, application and package instructions for desktop applications, scripts for deskside build and migration tasks, general company data, etc.  Our goal is to move as much information, except for financial control data, into the wiki format as the format will allow both because we find this system to be very effective for our small firm but also because by putting more information into a single repository we find that usage rates increase and more employees are able to find more types of data more quickly.

At CompanyX we have chosen the Use Case as our primary form of design documentation.  Use Cases are simple for all parties to create, consume and comprehend and provide the most important piece of documentation needed by the development team – a clear understanding of the customer needs.  Use Cases are divided into business use cases and system use cases and are designated as each within the wiki.  Business use cases come from the business analyst and include business process modeling.  System use cases come from the system analyst / architect and involve technical, system process models.  Comprehensive Use Case documents are a long term project resource since they reflect the way in which the system will be used or the way in which we believe that it will be used.  As such this documentation is much less likely to become outdated than more technical documents are wont to do.  In some cases, Use Case information close to a decade old has been beneficial in passing long term project knowledge between project iterations.

Beyond the use of Use Cases, which are often mentioned in conjunction with the Unified Modeling Language (UML) but are not actually a part of the UML specification, CompanyX makes selective use of UML diagrams and models for more technical aspects of the system design process.  The graphical nature of UML diagrams makes them naturally more difficult to store, catalogue, search and retrieve.  Because no system, of which we are aware, is effective at dealing with a store of UML diagrams we have chosen to use the wiki system again as a UML diagram repository alongside the Use Cases.  In some rare cases a UML Use Case Diagram might be attached directly to a Use Case but more likely a separate category of pages is created with carefully organized diagrams arranged as images on the wiki page with appropriate text on the page making the page as searchable as is possible.

While our use of UML is less extensive than Use Cases we find the more technical diagrams such as class diagrams and object diagrams to be the most effective.  Other types of diagrams would be used only rarely when a special circumstance called for a high degree of clarification.

UML diagrams are created using the Umbrello module of the KDevelop Developer’s Suite.  This package is available on all standard CompanyX Linux desktops.  Umbrello was chosen as the best UML modeling tool for our purposes after auditioning  several other tools such as ArgoUML, Kivio, StarUML and Poseidon.  For some special case needs Microsoft Visio may be required.  This is available on all CompanyX standard Windows desktop builds.

In addition to traditional design tools, CompanyX, being completely non-colocated, requires special attention to the needs of team communications in a collaborative development environment.  In order to support these forms of communications, which are especially critical during design phases, we have traditionally implemented instant messaging conference rooms and advanced voice telephony functionality that includes extending the organizational voice system to all offices and employee homes.  These tools are key for rapid communications but play only to synchronous or near-synchronous communication needs and do not address long-form discussion, discussion tracking or decision archiving.

To support these longer duration needs, we are suggesting that an online forum application be implemented that will allow teams to pose questions and ideas and invoke response and discussion both from the team itself as well as from the community.  Many open source forum and bulletin board packages exist to provide this functionality.  We have previously auditioned phpBB which is, most likely, the most popular package available in this space.  A new product, Vanilla by Lussumo, is also a recommended candidate.  Another potential candidate is Beast, a Ruby based forum.

Development Tools

Developers work day in and day out within development environments and because of the percentage of time spent within these tools it is very important that these tools be readily available and appropriate for the task.

Hardware

All developers should receive moderately high performance workstations which are more than adequate for software writing.  Hardware is effectively free in comparison to developer time and effectiveness.  The IT team keeps a standard desktop build design for hardware and software designed to be efficient and powerful whilst giving developers the flexibility that they need to be effective.

Because CompanyX has a highly “at home” workforce and due to the fact that our desktops need to be transported regularly it is useful to keep our machines as light and small as is reasonable.  The recommendation is to use the Hewlett-Packard Small Form Factor dx5xxx family of 64bit AMD Athlon64 and AMD Phenom based desktops.  These machines are available natively with 64bit Windows Vista to support the latest Microsoft-based development environments and are completely compatible with our Novell OpenSUSE 64bit development environment.  In some special cases, Apple Mac OSX or Sun SunBlade Solaris desktop may be requested.

CompanyX should make a concerted effort to rapidly replace all aging 32bit systems still in the field with 64bit hardware and software as all deployment platforms have already migrated to 64bit exclusively.  Development will be made more simplified if only a single architecture standard is in use.

Many companies choose to use full-fledged multi-processor workstations rather than high end desktops, such as are recommended here.  Those are very valuable and should be made available upon request but most developers, we believe, are unlikely to wish to have large pieces of equipment such as these in their home offices and they are unwieldy to transport making them a less advantageous choice compared to the less expensive small form factor (SFF) desktops.

Desktops should be outfitted with very fast, multi-threaded processors and extensive memory allotments to support rapid development, responsive environments and easy, local compiling.  Care should be taken if any deviation from the IT department standard hardware is made to ensure that the hardware selected is able to support multiple monitors as many desktop products, including units in the same lines as we use, are unable to do so without the addition of extra hardware which adds expense, support problems, time to deliver and complexity.

In addition to dedicated hardware, we have traditionally offered a remote desktop option for users working from remote locations or who just want access to an internal CompanyX managed Linux desktop sitting inside of the corporate firewall already configured with a large set of tools for development.  To enhance future developer flexibility it is recommended that this system be upgraded to newer hardware and software and designed to handle many simultaneous remote users nearly equivalent to the entire development team.  This remote desktop service should be upgraded to mirror our most current Linux standard builds to provide consistency between the desktops and the remote access services.  Also, currently Windows developers needing remote access to Windows development environments do not have a reasonable option to accomplish this.  The proposed solution is to add additional datacenter functionality that will support direct, remote access to developer’s personal workstations through a central accessibility portal.

Software

For Windows development, CompanyX has been quick to adopt the latest in Microsoft VisualStudio platforms moving to each one as it has become available moving from VisualStudio 6 through VS.NET 2002, 2003, 2005 and currently 2008.  VisualStudio provides a complete set of Integrated Development Environment (IDE), compiler collection and database design tools.  For our standard development package we believe that this set of tools is more appropriate for our chosen C# / .NET platform.

For UNIX development, CompanyX has chosen to use OpenSUSE Linux on the desktop and Red Hat Enterprise Linux on the server providing a nearly seamless set of tools.  Linux is our operating system of choice for most development languages including Java and Ruby.  Unlike on the Windows platform there is not a single, obvious source for all development needs on UNIX.  We have chosen the Eclipse IDE as our corporate standard for Java and Ruby development.  Eclipse is available on both Linux and Windows providing additional opportunities for platform independence.

On occasion, C and C++ develop is needed.  For this purpose we have decided to provide two options.  The first option is C support in Eclipse.  This is useful for most C tasks.  The second option is the Linux-specific KDevelop IDE which is well suited to development targeting the Linux KDE desktop and the Qt Toolkit.

Continuous Sustainable Velocity

CompanyX has long valued the importance of providing an environment where developers are given the best possible opportunity to innovate, design, create and write great code.  We believe that introducing excess overhead, including most deadlines, does not lead to better code or better innovation.  We want to be sure to provide our development teams with the best possible tools and environment in which to create while doing as little as possible to hamper their desire to produce.  Taking a page from the book of academic procedures, we see all of our development as being a part of “research and development” and encourage any flexibility necessary to encourage this type of behavior.

Continuous Sustainable Velocity, or CSV, is basically the aforementioned ideal.  The idea behind CSV is to create an environment and management structure to support a maintainable maximum development velocity without exceeding fatigue thresholds.  Traditional projects use deadlines as a means of pushing teams to finish code more rapidly.  Often this works but at the expense of slowed development after the push and a higher rate of bugs and errors.  This type of management behavior sacrifices long term gains in return for short term benefits and overall produces less code and what code is produced is of lower quality.  This is similar to Wall Street firms forfeiting long term financial growth in return for short term stock price inflation.  CompanyX does not have to answer to short term gain seeking investors nor does its software delivery model create a need for hard development and release cycles.  We are in a unique and enviable position to exploit these advantages.

CSV does not eliminate the potential use of iterations, that is not the intent.  The idea behind CSV is to eliminate deadlines which are set by management, marketing, sales, customers, etc.  All products are created on a “when it’s ready, it’s ready” basis.  Management works with development to determine “release criteria” which determines when certain milestones have been met.  Using time estimates from developers as approximate completion dates to provide managers with the ability to create meaningful “drop dead lines” for burndown task lists to mark when an iteration, version or milestone is complete provides us with a scheduling mechanism, but one very different from what most companies use.  In this way some amount of scheduling can be achieved without inflicting deadlines upon the development team.  Instead, deadlines are inflicted upon management.

Because of the Software as a Service model in use within CompanyX there is no need for a formal product release cycle.  Instead software can be released at the end of every iteration.  In many cases these releases are very small and changes could go unnoticed by customers.  If an important feature or fix is ready before an iteration is complete it can be released to production immediately.

CSV is designed to not encourage overtime but to encourage flexible time.  The idea is a sustainable pace but with the understanding every day cannot be the same and that without variety there is no way obtain a sustainable pace.  Employees are encouraged to work long hours when they want and to not work when they do not want.   CompanyX’s goal is to allow each employee to work in the manner in which they feel most productive.  For some this means eight hours per day, five days per week during normal, daylight hours.  For others this might mean working a constantly changing schedule or working in huge bursts of creativity followed by days of relaxing and not coding.  We recognize that people are unique and that everyone works differently.

General Development Best Practices

There are many industry accepted best practices for all types of software development.  Among these practices are code review, source control, unit testing, test driven development (TDD), continuous integration (CI) and bug tracking.  These are the primary practices which we will be targeting as most important to CompanyX at this stage and which, we believe, can be implemented and enforced quickly.

Code Review

Code review is an important part of any organization’s attempt to improve code quality at a line by line level as well as a way to create organizational integration by disseminating knowledge, technique and style between team members.  By having different team members regularly reading each others code it gives a perfect opportunity for learning, instruction and communications.  This is especially true for younger and more junior developers reading through the code of their seniors and mentors.

Code reviews have the obvious benefit of error reduction.  The idea here is not that code review will reduce syntax and compiler errors as those will generally be caught by automated tools (except for Python or Ruby where new code paths may be tested in production and not in testing) but that having a second set of eyes on the code will allow for logic mistakes to be caught or give an opportunity for someone to suggest an alternate approach that may be cleaner or more efficient.  Perhaps even more importantly, if a developer knows that she is going to undergo a code review she is more likely to make the code easier to read which naturally will also make it easier to support later.

Source Control

Source Control is an absolute necessity for any development environment.  Even teams of just a single developer should be using source control as it is so critical in maintaining records of changes and providing an ability to go back to previous source versions.  The larger the team, however, the more critical the  role played by source control.  Versioning is important but the ability to merge changes from several developers and to manage branch and trunk development is very important.  CompanyX has chosen to use the very standard Subversion source control system.  Subversion is open source, free and very widely used in the industry.  It has become the de facto standard over the previously popular Concurrent Versions System (CVS) which it was designed to replace.

Unit Testing and Test Driven Development

Unit Testing and Test Driven Development often go together as a pair of practices.  Unit Testing, often handled through a unit testing framework such as nUnit for C#, JUnit for Java and Test::Unit for Ruby, is a style of code testing that is done by testing the smallest possible piece of the code infrastructure, often the method or function.  Unit tests are a very important part of any standard refactoring procedure.  By writing the original code and testing against unit tests we have, in theory, proven that the atomic unit of the code is acting correctly.  There is still a possibility of error in both the mainline code as well as the unit test but this is far less likely and there is still the very real possibility of architectural or design mistakes but a great number of errors can be caught and eliminated early through the use of unit tests.  Unit tests, at a very minimum, force the developer to write the same piece of code in two different ways (a “positive” in the form of mainline code and a “negative” in the form of the expected results) which can have the benefit of forcing the developer to think differently about the problem and may result, in some cases, in faster and more creative problem solving.

The nature of unit tests makes them highly automatable allowing you to use them regularly, such as with each code check in, to determine that new code changes do not break previously functioning code or to verify that refactorings of existing methods continue to work as expected.  Unit testing increases the level of confidence in the code and makes integration much simpler.

Test Driven Development uses the principles of unit testing to suggest that instead of writing a method and then writing a unit test to verify the method’s validity that you can instead write a test, which will initially fail, that looks for the expected output of the to-be-written method.  Then the method can be written against the test until the test validates as successful.  The key benefit generally touted by proponents of this approach is that writing more tests, which TDD causes by its very nature, causes software development to be more effective.  Bug levels do necessarily fall compared to test-behind unit testing but productivity has been measured to be higher when tests are written first.  It is believed that this is caused by the thought processes and mental approaches to problems taken by the developers when they are writing the tests.

For CompanyX, TDD may be, at least at this time, rather too extreme for us to begin to implement but the use of unit testing, across the board on both new and old projects is very important.  Once unit testing is implemented and accepted then experimenting with TDD can possibly follow.

Continuous Integration

Continuous integration builds on the benefits of unit testing and source control.  The concept behind continuous integration is that each time code is checked in to the source control system, which should be a minimum of daily, that a full, automated build process is run creating, in theory, a completely working build of the package and then run the complete unit testing suite against it to verify that integrity has been retained.  Should any test or the build itself fail, the package must be corrected immediately to ensure that there is a working codebase for work to continue the next day.

CompanyX has not yet utilized continuous integration in our practices but this practice should be rolled out in conjunction with comprehensive unit testing frameworks.  Continuous integration most definitely benefits from having unit tests already in place but we are able to begin before unit tests are actually done allowing momentum to be achieved.  Popular continuous integration packages include Cruise Control, Cruise and Bitten.  We are assuming the use of Bitten as it is integrated with Trac, which we will discuss below.

Bug Tracking

Bug and Feature tracking involves a centralized error management system that provides a centralized location for recording bug occurrences, status, priority, etc.  Bug reports may be produced internally by developers or testers or externally by clients.  By having a central location for bug data managers are provided an opportunity to perform triage on the bugs and can provide developers with a clearly prioritized “to do” list of bugs.  This is important in keeping projects moving at maximum velocity as well as for minimizing impact to clients if bugs are appearing in production systems.

Bug Tracking systems are important for any number of reasons arising from the need to know where bugs are, when they have been fixed, by whom and how.  CompanyX has, in the past, auditioned products like Bugzilla and FogBugz for this very purpose but was very disappointed in the ability of these products to meet our rather simplistic needs.  We have decided to use the Trac product from Edgewall Software.  Trac is a popular bug tracking system that has an integrated continuous integration environment feature, Bitten, that can be implemented alongside of the tracking product.  Trac is already designed to integrate with a Subversion repository making it a very useful choice for CompanyX.

Project Selection and Prioritization

As CompanyX is a small firm with extremely limited development resources it is very important that we choose projects carefully and prioritize them as best as possible in order to ensure that projects with the greatest potential for success are chosen over those that are less likely to have large returns, have a very high cost to implement or carry a high risk of failure.

CompanyX’s model of software as a service means that our attention is always drawn, first and foremost, to maintaining the quality and integrity of our running applications.  Unlike traditional software companies for whom software in the field seldom represents a financial loss if new bugs are found but may actually realize a financial gain through support contracts CompanyX has a vested interest in keeping running applications running as smoothly, efficiently and effectively as is possible.  Unpatched, known bugs are very evident to our customers and may represent potential security or performance flaws that will impact our own infrastructure and support costs.  Running applications always take full precedence over any forward development project.

New, greenfield projects are started infrequently within CompanyX and choosing an appropriate project is an important decision that affects the company, as a whole, for any single project.  For a small firm like CompanyX which is profitable and maintains a relatively even level of developer and engineer resources versus workload in the pipeline possibly the most important selection criteria for a new project is availability of resources.

In most cases, at CompanyX, developer and engineering resources are not interchangeable commodities.  Developers and especially analysts are likely to be domain experts in a business area and projects are often selected based on the availability of these skill sets.  If new developer or analytical resources become available suddenly a new project may be initiated with very little planning in order to utilize these resources as quickly as possible.  In this way we are very opportunistic in our project selection process.  Cost / Benefit and other factors are generally subsidiary to resource availability.  This is not only because of the importance of resource availability to us but also because resource availability is an easy to determine factor while cost, benefit, MOV, ROI, etc. are extremely subjective and prone to very high rates of miscalculation and misjudgment.  Given the availability of a straightforward deciding factor, we believe that we will obtain best results by leaning heaving to the weighting of the known rather than the unknown.

CompanyX is very small and project prioritization is not a significant factor.  Projects are generally started when resources become available.  There are, of course, a number of projects that are internal or infrastructure related and can utilize any number of different resources and so these do require prioritization, but, by and large, project prioritization is handled naturally but the sudden availability of a market opportunity or availability of resources which are a good match for a particular project.

Prioritization of these projects is handled by executive management and is usually determined by expected ROI.  Internal projects tend to be of a moderately similar size and complexity so valuations are relatively straightforward and manpower reduction and competitive advantage are the primary ROI determinations.

Team and Project Organization

CompanyX, being a small firm, needs very little and relatively informal team and project organization.  Each organizational unit, being relatively autonomous, generally has a single development team assigned to it.  This segregation is intentioned to create an environment where each team feels empowered to self-organize as needed and as is appropriate for the nature of their project.  This lack of coordination also provides a more nature freedom for teams to choose architectures and platforms that most fit their needs rather than blindly following trends and directions forged by older teams and successful projects.

Steve Yegge of Google has stated that Google’s availability of great platforms can be a real boon for internal teams looking for resources but that it also squashes innovation by pressuring teams to implement large initial overhead systems as well as square-pegging technologies to work on Google’s existing platforms (i.e. BigTable, etc.) whether or not they are appropriate or advantageous just to justify the former investment in the technology.  This justification is clearly one that exists only on paper because if a platform must be used even when it hurts the project then it has actually introduced a negative ROI rather than a positive one.

Because CompanyX does not size teams based on a need to meet a schedule or deadline teams are chosen based on team member interaction, appropriateness for the team, technical needs, etc.  This team sizing method is designed around obtaining the maximum performance from each developer rather than picking an arbitrary speed for development and sizing a team until it is able to obtain that speed.  This sometimes costs us the ability to get to market as rapidly as possibly in favor of overall per-person productivity.  This impact is not as dramatic as it may seem, however, as innovation and high-performance teams are able to move quickly and keep pace with much larger teams that are built without regard for their team dynamic, communications, style and compatibility.

Typically within CompanyX teams range from two to five people.  We have found that smaller, more closely knit teams tend to obtain greater productivity per developer and that ideally teams should be in the two to three person range when possible.

Teams are only-lightly managed with self-organization as a high priority.  Obviously some management and direction is needed from outside of purely self-organization.  Direction is generally established by the business analyst assigned to each business area.  The analyst acts much like a manager as they are involved in deciding strategy, task priority, direction, etc.  The business analyst is not a personnel manager and no staff report to them.

Management is handled directly from the Director of Information Services.  This is the only dedicated management role in the CompanyX IT organization.  All IT staff report up to the Dir. of IS.  CompanyX is very fortunate to have the ability to avoid the bulk of politics that occur in most organizations simply because of our size.

Virtual staff is import at CompanyX as it is in almost any organization with more than a single team.  Because CompanyX is small, though, virtual staff is still a relatively minor consideration.  Technology expertise may be shared across teams but this tends to be quite casual.  The CompanyX wiki includes information including skills sets and areas of interest of employees so searching for skill areas is possible.

At CompanyX, it is the goal of our managers to make it possible for employees to work.  The idea is not that we have hired employees who need to be managed but simply that they need direction from time to time and, more often, need someone who buffers them from the non-development needs of the organization.  It is the role of the business analyst and manager, in this case the Director of IS, to eliminate distractions, unnecessary paperwork or other busywork and to provide a politics-free environment as conducive to working as possible.  CompanyX’s policy is that the manager is less in control of the team as a resource of the team.  We believe that if we hire the right staff and provide them the ability to do great work that they will so it is very important to keep out of their way and not stop them from being productive.

Project Management in the Continuous Sustainable Velocity Organization

In a traditional software development organization, project management is a rather heavyweight process which is heavily involved in the day to day work of the development teams as well as involved in the production of a large number of project artifacts stemming, generally, from a need to report project progress, status, details, etc. to upper management.  CompanyX does not operate with a need to produce project artifacts for upper management.

CompanyX management has the ability, and the responsibility, to pull project status by checking the project backlog and burndown lists.  In this way status is able to be monitored without needing the team to provide details back to management.  The monitoring is passive, not active.  At no point should the team stop doing actual, profitable work in order to produce paperwork for internal consumption.  Not only does this take time away from the development team and slow progress but it also would require management to spend their time reading status reports rather than being productive themselves.

Because we do not market products before they release and do not have deadlines to meet in delivering products to our customers the needs of our organization are much simplified over more traditional software firms.  This gives us an additional advantage in that we can spend more time innovating and producing software at lower cost making it easy for us to outpace our competitors to market.

Some pieces of the traditional project manager role are played by the business analyst.  Important decisions such as when a feature is complete or what level of functionality is acceptable are determined by the business analyst.

The PMBOK specifies many tasks which are a part of the project manager’s role but a large majority of these are simply assumed by the creators of the PMBOK and are not actually necessary.  At CompanyX, our model is continuous development so we do not “time box” development work into a project that has a definite end date.  We cannot budget by release because we do not know when a release will be completed or what it will contain.

Instead of traditional budgetary means, CompanyX establishes an ongoing development budget that is defined in a cost/time manner, such as $65,000/month.  This cost includes the salaries of the project team for that duration, an estimate of the months cost of electric, HVAC, Internet access, software, hardware, networking, telephones, chairs, desks, etc.  Using this number, which is extremely accurate and varies in predictable ways, project budgetary planning can be carried out based upon the needs for the project to continuously outperform its cost by way of its revenue.

Instead of determining a project’s total cost up front we determine the ongoing cost of a project and determine if we are prepared to support the ongoing cost of development and support.  This still requires budgeting and financial planning but the manner in which it is done is far more straightforward and intuitive.  It is also far more honest than planning done in many organizations as the ongoing support cost is both difficult to fudge (if resources are assigned, cost is assigned) and total project scope is not a part of the picture. Scope creep, which is controlled by the business analyst, will affect development duration but will not impact the budgetary process.

There is no cause to make early determination as to scope before the project has begun.  In order to determine a project budget a normal firm will need to decide upon scope, features, architecture, etc. before the project budget has been approved and, therefore, before the project itself has begun.  CompanyX avoids the costly and error-prone process of making scope decisions when far too little is known about the project and, instead, determine a velocity desired and/or budgetary need that can be withstood and allow the business analyst along with the development team to determine feature priority and to decide when the product has enough features completed to allow it to move from purely development into production.

Hiring Practices

One of the, if not the, most important endeavor undertaken by the staff at CompanyX, or arguably any company, is the accumulation of staff.   Because CompanyX focuses less on processes, procedures and documentation and more on team effectiveness, innovation and employee satisfaction we are far more affected by the quality of the staff that we hire.  This greater employee dynamic can be seen as a benefit or a deficit.  If we hire great staff then we empower them to be creative and productive. If we hire poor staff we empower them to do nothing.

CompanyX has always been and must remain very aggressive in its stance on hiring and retaining the best IT talent in all IT areas including engineering, development, administration, architecture, support, etc.  We have found that by the very nature of our management practices and our use of continuous sustainable velocity that we have naturally managed to maintain staff once they have been acquired.  Most ambitious and talented IT professionals, especially developers and designers, are thrilled to find an environment so dedicated to respecting their professionalism, providing a framework for their own learning, growth and production, peers that are passionate about their field and the projects on which they work and a lack of red tape separating them from the work that they wish to do.

Far more difficult than retaining great talent is locating and identifying that talent in the first place.  At CompanyX it is very important that we find not only great skill and experience but also great drive and passion.  CompanyX is a company focused on innovation and to achieve this we require staff that will constantly seek out and discover new avenues for exploration.

Our first means of seeking out great candidates is to simply not seek candidates.  CompanyX has always hired exclusively through people seeking out an opportunity to work with CompanyX and through word of mouth.  This drastically reduces the chances of hiring someone who is desperate for work rather than seeking a wonderful development environment.  We do not want staff that are simply happy to have a job which will pay the bills but want staff who are actually excited about the work that they get to do and would be very unlikely to leave the company because almost no other company offers the exciting, driven environment that CompanyX offers.

When in the hiring and interviewing process for new candidates it is very important that we do not stress specific technical skills.  Any candidate may have a select number of highly technical skills for which we test which proves nothing.  Technical testing also requires us to test a candidate against a skill set for which we assume knowledge will be needed for the job.  This is a short-sighted methodology based around an immediate need and not the long term growth of the company.  No technology is so complex that we must require that a candidate bring that existing knowledge with her.

Far more important than a candidate possessing an immediate technical skill is that the candidate has a set of technical skills – any set.  What CompanyX actually cares about is the ability for a candidate to grow and adapt.  The technical landscape is constantly changing and the skills useful today will not be useful tomorrow.  What we are looking for is a candidate who will embrace tomorrow’s skills and learn them easily.  Otherwise an investment in an employee is an investment in stagnation.  If a candidate earns a job because of a single skill set then that candidate is also more likely to leave later when we move forward to look for “comfortable” work at a firm using legacy technologies.

Even within a single technology for which we would choose to interview, Java for example, there are a range of specific technical skills necessary to be useful on day one and expecting any particular candidate to use the exact same set of technologies and approaches which we are currently using would limit the potential candidate pool dramatically while likely eliminating candidates who could pick up the skill in a weekend and outperform most candidates in a week or two.

For example, within Java candidates may be used to platforms such as Weblogic, Websphere, JRun, JBoss, Spring, Tomcat or WebObjects.  They may work with different databases like Sybase, SQL Server, MySQL, PostgreSQL, Firebird, SQLite, BDB, Oracle, Informix, DB2, Derby, etc.  ORM packages like Hibernate, iBatis and others.  They may use EJB 2, EJB 3 or POJO approaches.  They may run on Windows, Mac OSX or UNIX.  Perhaps they use Swing to make desktop applications or only write component services.  The variation is far too wide to expect that the right fit for CompanyX has been working with the same technology stack as us.  If we do find a candidate working with the exact same stack then we risk that that candidate won’t be bringing in fresh approaches and ideas which is some of the greatest value available from new candidates.

Along with technical prowess we also are very concerned with finding candidates who are extremely passionate about technology and are driven not just from a career perspective but also from a desire to excel, to create, to grow and learn and to make, manage or design amazing software products.  A candidate with drive but little experience is more valuable than one with much experience but that is not concerned with future personal growth.  Working at CompanyX is a lifestyle of technical excellence.

Along with the standard hiring process we must also consider the opportunities that lay in an intern or co-op program.  Interns may come from any background including current technical IT work, academia, high school, etc.  In an intern we look primarily for drive.  Drive is most recognizable in intern candidates of any age who are willing to work on their own projects, at home, in their own time and have been doing so long before becoming interested in working with CompanyX.  This same drive is sought after in any candidate, not just one at an intern level, but unlike more senior candidates who have an opportunity to differentiate themselves through career experience, skill set areas and previous projects, interns are viewed almost exclusively through their “at home” experiences.

Recognizing two important facts about academic backgrounds is also important in picking candidates.  The first is that many of the best candidates will not have attended college or university because they were too busy already being in the field.  The second is that if we desire to add academic experience to a candidate we can easily do that after they are hired; there is no reason to require this before the hire process begins.  A tertiary issue also exists that the majority of collegiate IT and CS programs are both far too easy and often avoid core topics that would be expected, and in so doing lessen the potential value in judging a candidate by their academic background even when it appears to be pertinent.  Because of these two facts it is CompanyX’s policy to disregard academic experience except in cases where the candidate has undertaken the process of obtaining a degree after having entered and while remaining active in the field.  Using university studies as an excuse for absenteeism from the field is not counted as positive experience time but as a resume gap.

Outsourcing is an important topic for companies like CompanyX.  Because we have no centralized, physical location we are well suited to both outsourcing and offshoring.  We will not, as a company, consider outsourcing labor to another software firm because we both believe that other companies are highly unlikely to be as competitive as we are leading to only higher costs and lower output and also because our value is in creating software and managing IT solutions ourselves.  It is these key tenants on which CompanyX builds its reputation and creates its value.

Using labor external to the United States, however, as employees is another matter.  CompanyX is officially nationality and locale agnostic.  We have no reason not to hire from international locations.  Even timezone issues are of little importance as our current staff regularly works floating hours and often works weekends or at night.

Quality Assurance and Testing

At this time, CompanyX does not utilize a dedicate QA and/or testing team for our software.  All of our testing is done by the developers and analysts and is done using informal methods.  This procedure is far from being ideal and leaves a lot of room for critical improvement.  This is an area in which CompanyX should focus heavily in developing a team and process for standardized testing.

The first step in preparing for formal testing is the creation of formal, separate testing and production environments.  At this time CompanyX has development environments running on desktops throughout the company and all testing is done directly at that level.  We are currently in the process of building out a dedicated testing datacenter which will be used to run pre-production code in a live setting for serious integration and testing work.  We currently have a Solaris testing environment built and  a Windows 2008 environment is already in process.

After we have a dedicated testing environment, our next step is to begin to build a dedicated testing team.  We will need professional testers with testing experience and knowledge who can help us to build a testing practice in the same manner in which we have built a development practice.  Implementing both white box and black box testing, testing automation beyond unit testing, interface testing, etc.

We have been fortunate that our small, tightly integrated user base thus far has been resilient to bugs as almost all of them clamor to be involved in early testing but this will change.  Our development methodology tends to introduce a low number of bugs, but we should be striving to lower that number as far as possible.  As our products are used by a larger and larger set of clients this will become more and more important.

Developer Mentoring

Mentoring is an important element in any professional development environment.  Mentoring adds organizational value for junior staff members as it gives them an opportunity to learn and grow with direct access to senior staff members with real world experience and value unobtainable through other avenues.
Mentoring is also important because it provides an opportunity for knowledge to flow back through the organization.  There are few mechanisms through which experience both personal and project or technical get disseminated, in a meaningful way, through an organization and one of the most powerful means of doing so is through mentoring.

The process of mentoring also helps to build stronger inter-personal ties within the company.  People on different teams or in different technical areas can use mentoring relationships to build networking ties and to become more acquainted with the organization and its resources.  Mentoring is significant in cementing corporate culture when new staff come on board.

Project Lifecycle Issues under Continuous Sustainable Velocity and Software as a Service Models

In a CSV and SaaS organization we face particular challenges less common in more traditional software companies in determining exactly what constitutes a project lifecycle and how that relates to its team and team members.  When software is delivered as a service it tends to take on a wholly different type of life than does software that is boxed and shipped.

Traditional box and ship software, or other types of software typically delivered in a “versioned” way, has a natural project lifecycle associated with each new version of the software.  Often teams will be known by their version such as the Word 2003 and Word 2007 teams at Microsoft.  This allows for projects to have a definite initiation point and a clear, or mostly clear, finishing point.  Some amount of ongoing support and patching generally continues but is minor in contrast to the project itself.

When working with Software as a Service and especially when doing so without the benefit of regular iterations there is no clear division between software development and its ongoing support.  For example, ProductY, our flagship product, ran as a project for approximately eight years.  During this time it went through initial development, ongoing support, new features, performance enhancements, stability improvements, underlying platform migration and more.

After eight years the ProductY codebase was replaced by the ProductY 2 codebase.  This new codebase we expect to go through a similar lifecycle but lasting approximately twelve years.  We anticipate the retirement of this codebase by 2020.  In this case, the team responsible for both codebases is the same.  Not one person changed between the original ProductY and ProductY 2.  From the perspective of the team, ProductY 2 is simply a natural continuation of the original ProductY and part of the lifecycle involved the migration of clients from one system to the other.

When applications go through lifecycles of this nature and on a timescale of this magnitude it is exceptionally difficult to differentiate between different phases of the project.  Is initial development until beta one project and then ongoing support another?  Sometimes though there is far more code developed after beta than before.  Are the two ProductYs separate projects?  Most likely that is true.  That is the most appropriate project divider that we have identified.  Still this is a very long time for the concept of a project to survive.

Because of these issues, CompanyX is faced with almost not even having the ability to identify teams by project at all.  This is difficult as the thought processes of developers, and humans in general, invariably brings them to the project.  At this time we have not determined the correct answer to this problem but we do have an approach that is, thus far, working while still being a work in progress.

Instead of organizing by project, we have been organizing our teams by application or feature that is a long-term, externally identifiable entity.  Therefore, to us, ProductY and ProductY 2 are the same entity as they are a single product to our customers.  A separate hazardous waste product, while similar and sharing the business unit, would be a separate identifiable entity and would be a separate “project” internally to CompanyX.  This approach, at this time, continues to function but also continues to evolve.

The overall lifecycle of one of these “projects” is very fluid.  Project initiation is very similar to a traditional project approach used by any software firm.  Initial requirements gathering and basic design is done and then code development commences.  Once a potentially useful level of development is deemed to be complete by the business analyst and testing is complete the application is released into production.

Once an application is in production there is no guarantee that it will be used by customers.  At this point we generate a basic sales and marketing framework such as a web site, support contacts, etc.  The appropriate marketing manager determines if the application is marketable in its current state.  If so then marketing begins to attempt to garner interest in the product.  If not, application development continues as before attempting to complete more features and increase functionality.  The development process is not driven by marketing except in rare circumstance nor is marketing driven by development.  Customers seeking our products always have a means to find them even when marketing does not feel that the product is feature-complete enough to justify marketing expenditures but new customers are unlikely to be located until marketing sees the product as more mature.

Regardless of the involvement of the marketing team, development continues on in the same manner.  The purpose of the initial lean release is not necessarily to get early adopters but to give customers who are anxious to use our products every opportunity to do so, to get earlier real-world feedback and to put the stake in the ground as to the launch of the codebase.

Throughout the lifecycle of the code development continues.  In some cases development may slow as the product ages and becomes more mature.  There is definitely a need for more development while ramping up a product than in ongoing support.  However, unlike 37 Signals who run a similar model, we are not targeting lean software as our primary market but extremely powerful and robust software with powerful business modeling potential and this gives us more opportunity to mold our products long after their initial release into the market.

In addition to feature growth, old code bases will often see refactoring, bug fixes, performance enhancements, library or platform upgrades, interface updates, etc. which supply a large amount of continuing development work long after initial design and release pieces have been completed.  At some point, as we saw with the ProductY product line, a new codebase will likely be generated and at that point development begins anew with a blend of fresh approaches and new designs along with new technologies while drawing from former knowledge and experience.

In the case of the ProductY, the initial system was a web-based Microsoft DNA architecture application running on VBScript/ASP in the Windows NT4 and SQL Server 7 era.  It relied upon Internet Explorer 4 as its client side platform and used a small VB6 client component to handle hardware integration.  ProductY 2, while providing identical functionality to the customer, does so using C#/ASP.NET on Windows 2003/2008 and MySQL with a single, non-web based client side C# application that handles user interface and hardware integration concerns along with local caching with a client side database.  The two architectures and approaches are very different but the functionality is the same.

Packaging and Deployment

CompanyX has a fairly unique need for packaging and deployment of its software.  As all of CompanyX’s software is deployed internally to our own platforms and our own systems we have an advantage over most firms in that we have little need for broad platform compatibility testing nor do we necessarily have a need to package our applications for easy deployment.

It is tempting, given the simplicity with which we can have the development teams perform their own deployments through a manual process, to completely forego the entire process of performing application packaging.  This makes sense in many ways.  It does, however, present a few problems.

The first problem that a lack of packaging creates is an over-dependence upon the development teams.  The developers themselves should not be required to be directly involved in the release of a new package, at least not with the physical deployment of the package.  By having a package created it allows the system administrators to perform application installations easily and uniformly.  This is not necessarily advantageous on a day to day basis but when there is an emergency and a server must be rebuilt quickly having the applications ready, in different versions, for the admins to apply to the environment without developer intervention is very important.

The second problem that arises without a packaging process is that software tends to become extremely dependent upon a unique build and deployment environment.  Shares to the environment from a system administration and engineering perspective may inadvertently break packages.

In a Red Hat Linux environment, for example, a customer deployed package may have a library dependency unknown to the system administrator.  That library may be updated, moved or removed, breaking the software quite accidentally.  If that same software were to be properly packaged as an RPM (Red Hat Package Manager) package then the RPM system itself would check dependencies system-wide and block the removal or changes to other packages needed by the application software.  When installing the application software a tool such as YUM (YellowDog Update Manager) would also be able to satisfy dependencies automatically rather than sending the system administrator off to find packages supplying necessary libraries.

For a company like CompanyX, our packaging needs are very simple.  Packaging formats such as RPM for Linux (Red Hat and OpenSUSE) and DStream for Solaris (aka Solaris Packages) provide important failsafe mechanisms to make installing fast and simple while protecting the software from inadvertent damage.  They also make version control and system management faster and easier.  More advanced packing, such as graphical installers, would be overkill in this environment and would, in actuality, make the deployment process more cumbersome considering that all potential deployers of the software are skilled IT professionals and not end-users and customers.

Management Reporting

Earlier I spoke about the availability of project backlogs and burndown lists as important tools for management to keep an eye on the status of projects without needing to interfere with the project team itself or to send agents to collect data on the team.  In many cases this is not enough information for management or for the rest of the company.

Currently CompanyX works on the basis of the basic backlog and burndown lists along with word of mouth, wiki posts, email distribution lists and a bidaily status and design call in the late evenings.  These tools, however, are not scalable and do not provide adequate reporting to parties outside of the direct management chain.  More information is needed but it is critically important to keep management from becoming directly involved and interfering with the teams and the projects.

The first status report which should be created for management is the build report from the continuous integration system, mentioned above.  Management should get a report, generated every time that the system is run, that details the status of tests passing and failing.  This allows detailed progress information to be noted without requiring any additional input from the team itself.  This is a good utilization of best practice processes that will already be being put into place.

The second important reporting tool, important for more than just management but for the entire organization, is developer, analyst and team blogs.  Having a project/team blog that is updated by the business analyst and/or development lead at least once a week gives insight into the projects for other teams, interested parties and, of course, management.  By providing all employees with their own internal blogs there is a system for personal status reporting and organizational information gathering.  Tools that may work well here include SharePoint, Alfresco, WordPress and Drupal.  Anyone in the organization would have the ability to subscribe to an RSS feed or to visit a website to find the latest information from any person or project.

In additional to traditional blogs which are great for technical posts and project status updates, the more recent concept of the microblog, made popular by Twitter, may also be beneficial to the organization.  Microblog posts are generally limited to 144 characters per post, just enough for a sentence or two.  This format could be useful to allow developers to post their thoughts, locations, status, needs, etc. throughout the day.  CompanyX has even written our own client for making these types of post easily from the command line while working in UNIX systems which is extremely handy for developers to effortlessly make quick statements to the company.  For internal use a microblogging platform server such as Laconi.ca would be appropriate, not a public service like Twitter or Identi.ca.

As an example of how microblogging could impact development, a developer might post “Stumped on a C# RegExp.”  Someone with extensive C# regular expression experience might see the post and instant message that developer offering some assistance.

Internal Training and the Development of Practice

The idea of Practice within the IT organization is a critical one.  By Practice we are referring to the idea of approaching a discipline as a true professional, perhaps even more appropriately, as an amateur in the classical sense.  The idea of Practice is, unfortunately, often a nebulous one, but one that is key to  the CompanyX approach to software development as a whole.

Practice refers to the holistic approach to the technical discipline whether that discipline is Java programming, Linux system administration, Windows desktop management, project management, business analysis or software architecture.  This means that we must look at the individual technical area as an area in which we can learn or develop techniques, strategies, best practices, etc.

Instead of approaching each technical area as just another tool in the tool belt, we look to encourage serious study of the technical areas which may include reading books, studying in a collegiate setting, joining user groups, attending vendor seminars and events, becoming active in online communities, writing and publishing papers, independent research and more.

Internally we support general educational opportunities through support of collegiate academic programs, unlimited book budgets and the Borders Corporate Discount program, internal training, mentoring programs and organized trips to vendor events, seminars and support groups.   Most importantly, we believe, it is important that we promote a culture of education and not merely offer opportunities but make learning an important part of everyday life within the organization.  We may also want to pursue projects such as having a reading group which meets to discuss readings in important industry publications or discusses seminar talks and panel discussions.

Software Development Best Practices and Standards

Even in an organization such as ours it is important to retain practices and standards throughout the organization.  Standards apply in many areas.  Applying standards, when possible, is an important factor in making code readable and rapidly adaptable when working with multiple developers.

Two areas in which coding standards need to be applied, across the board, are in naming conventions and code style standards. Both of these areas fall outside of technical areas of expertise and contribute to communications.

For a firm the size of CompanyX, developing internal style and convention guides would be extraneous overhead and unlikely beneficial to the environment.  Instead, we have decided to standardize upon the widely accepted Cambridge Style standards.  These are well known standards used not only in business but often in academia and widely used in professional publications.

Using these standards assures us that potential candidates are, at the very least, familiar with the standards even if they have not used them directly, will recognize the patterns easily and will write in a common style for the industry when publishing as well.  By using such common styles CompanyX can leverage the benefits of style and standard without unnecessary overhead and expense.

Style standards apply both to the “beautification” of code including things like the appropriate use of white space, placement of braces, use of multiple elements within a single line, etc. but also to naming conventions for variables, methods, classes, objects, etc.  Through the proper and standard use of such styles and standards programs become more readable, portable, extendable and maintainable.  The ultimate purpose is, we believe, to support the concept of Literate Programming as best stated by the legendary Donald Knuth:

“I believe that the time is ripe for significantly better documentation of programs, and that we can best achieve this by considering programs to be works of literature. Hence, my title: ‘Literate Programming.’

Let us change our traditional attitude to the construction of programs: Instead of imagining that our main task is to instruct a computer what to do, let us concentrate rather on explaining to human beings what we want a computer to do.

The practitioner of literate programming can be regarded as an essayist, whose main concern is with exposition and excellence of style. Such an author, with thesaurus in hand, chooses the names of variables carefully and explains what each variable means. He or she strives for a program that is comprehensible because its concepts have been introduced in an order that is best for human understanding, using a mixture of formal and informal methods that reinforce each other.”

Process Management Standards

Many software firms use process management standards such as the Capability Maturity Model Integration and ISO standards to control, improve or monitor processes.  No formal process improvement program or strategy is currently in use within CompanyX.

Capability Maturity Model Integration

The Software Engineering Institute at Carnegie Mellon developed the Capability Maturity Model Integration system with the idea that this model would be utilized by organizations seeking to build a framework of continuous process improvement.  The CMMI is a set of guidelines as to what processes should be implemented but not how those processes should be implemented leaving the actual implementation up to the individual organization.  This is good in that it allows for a great deal of flexibility when it comes to the actual usage of this system with a unique company.

The ideas behind CMMI are almost exclusively targeted at very large, non-IT organizations.  These ideas are not necessarily bad but assume a large amount of process overhead which is not present in many companies.  Introducing process where it is not needed just for the sake of managing and improving said process is unlikely to be an effective use of available resources.  Much of CompanyX’s management approach has traditionally been focused around reducing management and process overhead in order to make the company more efficient, agile and responsive while reducing costs.  The introduction of a process such as CMMI could easily double operational expenditures even without pursing official certification.  It is also likely to reduce agility and increase ongoing development cost by reducing development team efficiency.

Lean

Lean Software Development is an application of the principles of Lean Manufacturing to the software development process.  Lean is simply an attempt to eliminate waste from the development (or manufacturing) process and is directly related to concepts such as Just in Time Manufacturing – the process of having parts arrive at the factory just as they are needed for assembly reducing storage cost and leverage the time-value of money.

The concepts of Lean echo (or predate, in fact) concepts widely accepted in software development such as YAGNI (You Aren’t Gonna Need It) which encourage developers to save features that they cannot guarantee will be needed until they can.  This often reduces the total amount of effort needed to make a product since useless codepaths are less likely to be constructed.  Extra code not only costs at the time of creation but it also costs later to maintain and generally reducing system performance.

While Lean is often associated with process models and systems such as the CMMI and Six Sigma, Lean Software Development is more appropriately an Agile development methodology and is organized as such by its designers, Mary and Tom Poppendieck.  Lean is a relative newcomer to the methodology space appearing as recently as 2003 and appears to be mostly a repackaging of most core Agile principles without the addition of many of the “scarier” practices such as pair programming in XP.

For any organization utilizing an agile process, Lean seems to be less of a firm methodology and more of a repackaging of obvious and common sense items.  Lean’s greatest advantage, most likely, is for organizations who are caught in the “manufacturing” mentality to pick up a truly Agile framework and see it as “emulating Toyota”.  Lean may have the ability to be sold to managers who otherwise won’t accept processes without management buzz words attached.

Six Sigma

Six Sigma is a manufacturing process improvement program initially development by Motorola in 1986.  Six Sigma and Motorola processes are exceptionally well known to businesses in Western New York as many of them, including then Motorola CEO George Fisher, were brought to Eastman Kodak.  These manufacturing-designed processes and management when applied to Kodak, a research and development organization were used to drive out innovation, cripple agility, alienate the workforce and effectively dismantle the company.

Six Sigma itself is a manufacturing improvement process that focuses upon a feedback loop of quality improvement following on the tradition of many standard quality improvement procedures.  In fact, Six Sigma has been criticized for being nothing more than marketing fluff applied to obvious principles long established in the industry.  Six Sigma is often associated with other initiatives that have been labeled as management fads such as Total Quality Management and Reengineering.  These processes and their ilk were lambasted by Scott Adams in his landmark business book The Dilbert Principle.  Many of these fads and processes have come to be the face of poor management today even though mostly they have faded towards obscurity.  Six Sigma remains more strongly and may not constitute a fad since it is simply a repackaging of traditional concepts.

The biggest problem with Six Sigma is that it is a manufacturing methodology that relies upon repeatable processes and measurable error rates to achieve its goals.  As software development has neither of these things Six Sigma is meaningless in this space.  Even Eastman Kodak, a company whose fortunes lay mostly in their research and development, saw Six Sigma and heavy manufacturing processes erode their competitive advantage when their organizational focus was shifted to low-profitability and easily off-shore-able plastics manufacturing rather than pharmaceutical and technology research.  Even Motorola, the poster child of Six Sigma, saw its fortunes dwindle compared to its chip making competitors and the market as a whole once their focus shifted from new product development into manufacturing.  In first world markets, any process that can be measured and improved through a system such as Six Sigma should be earmarked for third world consideration.  In those markets, the principles behind much of Six Sigma should probably be applied to their manufacturing processes as there is much value when the core business is actually manufacturing and not product development and innovation.

The mistake often made with Six Sigma outside of manufacturing is the belief that all processes, even those that are inherently creative, can be meaningfully measured and quantified.  Few people would think of applying the principles of Six Sigma to painting a picture, composing a symphony, writing a novel, acting in a play, dancing in the ballet, etc.  Why?  What about an individuals expression through dance is less quantifiable than the creation of software?  Software, according to Donald Knuth, is literature and, according to Paul Graham, the processes making it are closer to art than engineering and even engineers would be appalled to find their engineering processes being measured and quantified as if they were factory workers – interchangeable cogs in a giant machine.  If companies believe that design processes are interchangeable and replicable why are they not automating them.  Clearly the end products of mechanical engineering and software development are both delivered as computer data files and so would be perfect candidates for total computer automation.  Why has this not happened –  Because both of these processes’ values are in their creativeness and not in their repeatability.  Therefore Six Sigma has no meaning in this context.

Lean Six Sigma

In theory, Lean Six Sigma is the application of Lean’s Agile development principles to a Six Sigma process improvement environment.  In manufacturing circles this is an obvious marriage.  In software circles, however, Lean does not in any way explain how Six Sigma can be applied to the creative process of software design and writing.  Even Mary and Tom Poppendieck, the creators of Lean Software Development, when speaking about Lean Six Sigma, are only able to explain the application of Lean in a software environment and neglect to explain the use of Six Sigma.

Summary

There are two things that often result from the application of process controls in an organization.  The bad is the creation of unnecessarily heavyweight or even possibly harmful process that encumber or possibly disrupt productivity and increase overhead costs making it more difficult to achieve profitability.  The good is the codification of common sense principles.  What is difficult is to avoid the first while leveraging the second without letting the second become the first.

Few process management advantages come from applying unintuitive principles to software development.  Sometimes principles which are obvious after you are aware of them, such as YAGNI, are non-obvious or even counter-intuitive to novice developers, managers and teams.  In cases where the organization is not aware of these principles at an organizational level and needs to apply them down to teams it is generally effective to codify them.  However, we do not believe that it is valuable to codify principles which are already understood and obvious.

Every organization has a different level of organization knowledge in software development.  The larger the company the great need for process control and codification.  Smaller companies have more opportunity to transmit good processes through the idea of Practice learning without the need for formal codification.
By transmitting through Practice we have an opportunity to teach, learn and grow rather than by enforcing.  Transmitted knowledge gives everyone an opportunity to evaluate, internalize and comprehend procedures and, most importantly, it gives everyone the right to question them.  If a process is codified within the organization then only a select few people are likely to have any real ability to question the applicability of a process or procedure and innovation may be stifled.  Few processes are appropriate all of the time and so teaching everyone when they are useful and when they are cumbersome empowers everyone to be as effective as possible.  Codification takes that decision out of the hands of the people with the most direct ability to determine the appropriateness of a procedure within a particular context.

Organizations blessed with highly skilled staff need very little process control as the staff will generally regulate themselves and follow meaningful processes on their own.  Some process control is still necessary, in this there is no question.  A minimally skilled organization, on the other hand, will require a great deal of process control to allow them to obtain any degree of productivity at all.  This is the fundamental theory behind a great many business and management processes – the concept of “managing to the middle”.  Managing to the middle means that instead of acquiring and training and empowering great staff the company has decided to acquire budget resources and using heavy process control to obtain some level of production from staff that would be mostly dysfunctional with those processes.

At CompanyX our base principles rely on the fact that we acquire great, motivated resources and empower them for innovation and creativity.  This has been said many times in this text and it cannot be stated enough.  Every management decision made at CompanyX must be done in the light of the fact that we trust our teams, and we believe in our teams.  CompanyX is not a code manufactory.  CompanyX creates literature.  It creates art.  It creates innovation.   Therefore it is imperative that we refrain from utilizing process control beyond that which is absolutely necessary and, whenever possible, rely upon the foundations of Practice and professionalism to establish working processes that are themselves flexible and mutable.

Risk Management

In software development, risk is a constant presence that must be addressed.  Risk comes in many forms such as choosing to enter an inappropriate marketplace, finding a new competitor after a project has begun, poor design, misidentification of business needs, bug-ridden software implementations, failed vendor support for key platform components, regulatory changes, unforeseen technical difficulties, team malfunctions, key employee health issues, lack of internal political support, etc.  Risk is unavoidable and it is by facing risks that we are presented with an opportunity to move into new, unproven territories.  With great risk comes great rewards.  But nothing is without risk, even an absence of action does not truly mitigate risk, in fact it is likely to exacerbate it.

“If you don’t risk anything, you risk even more.” – Erica Jong

Risk Identification

The first step in dealing with risk is identifying potential points of risk.  For CompanyX we know that our most significant points of risk come from personnel dynamics which include health issues, availability, etc. and post-production market uptake.  There are many other risk areas but to these two we are more susceptible than many companies and need to be acutely aware of the implications of these risks.

Before any project begins a detailed risk assessment process should be followed.  This procedure should include development lead, design lead and financial management at a minimum.   Our organization fortunately is able to avoid some areas of often high potential risk because of our minimal political turmoil, abundance of project support and lack of risk aversion.  For example, CompanyX has never been forced to abandon a project because of a withdrawal of internal executive sponsorship which may plague even the most successful of projects in other companies.

Risk Assessment

Once a comprehensive set of foreseeable and reasonable risks are collected then we must assign a likelihood of occurrence and a level of impact quantity to them.  Obviously this measurement is highly subjective.

Unlike a larger firm which may be able to spread risk out widely between projects and hedge high risk projects with low risk projects we are stuck in a position in which we must put our few eggs into very few proverbial baskets and our organizational risk to a single failed project is quite high.

Risk Mitigation and Handling

Once risks are understood there are four options for dealing with risks.  Risk avoidance, reduction, retention and transference.  Each approach must be considered and decided upon by the risk team.  In general, however, it is in the interest of CompanyX to practice risk retention when possible within the software development practice because our ability to withstand failures internally is generally better than is the industry average causing other forms of risk mitigation to be more costly by comparison.  My retaining risk we have a greater opportunity to leverage our internal skills more effectively.  Obviously this does not apply when risk can simply be avoided.

Summary

CompanyX has pursued process management very diligently in the past and many of our processes and procedures reflect a dedication to research, customization and independence, but we still have a very long way to go.  There is much that we have to learn from the industry and many best practices that we still need to apply.

Building a strong process supporting strong projects requires constant diligence, reevaluation and reassessment.  There are always areas in which improvement is possible and must be pursued.
In this document we have identified areas of improvement that should be addressed such as our lack of organized testing and our lack of support for a testing Practice and a need to implement more formal risk assessment procedures.  New software systems such as continuous integration and discussion forums needs to be installed and made available to the development teams.

In the past year we have been making many strides in the right direction and improvements are very visible.  Perhaps our greatest achievement of the past year is the use of the wiki format for storing procedures, design documents and other forms of organization knowledge in a repository that is not only highly available but also easily searchable.  It is very exciting to see how this resource will be adapted for use in the future.

Whatever process improvements we seek out in the future it is imperative that we remain focused upon the fact that our value lies in our staff and that any process which stymies their desire to create or hampers their ability to remain productive is dangerous and should be avoided.  If we undermine our own value then we have lost the very resource for which we were hoping to be able to manage through process control.  Like any form of management, process management has more ability to damage than it does to accentuate and care must always be taken.

Process management must be a tool for enabling innovation and creativity because it cannot be a tool to create these things.

References

Wells, Timothy D. Dynamic Software Development. New York: Auerbach, 2003.

Brooks, Frederick P. The Mythical Man Month: Essays on Software Engineering.

Graham, Paul. Hackers and Painters: Big Ideas from the Computer Age. Sebastopol: O’Reilly, 2004.

DeMarco, Tom & Lister, Timothy.  Peopleware: Productive Projects and Teams.  New York: Dorset House, 1987.

Rainwater, J. Hank.  Herding Cats: A Primer for Programmers Who Lead Programmers.  New York: APress, 2002.

Spolsky, Joel.  Joel on Software.  New York: APress.

Spolsky, Joel.  More Joel on Software.  New York: Apress, 2008.

Mason, Mike.  Pragmatic Version Control: Using Subversion – The Pragmatic Starter Kit Volume 1.  Raleigh: Pragmatic Bookshelf, 2006.

Hunt,Andrew & Thomas, Dave.  Pragmatic Unit Testing in Java with JUnit – The Pragmatic Starter Kit Volume 2.    Raleigh: Pragmatic Bookshelf, 2003.

Clark, Mike.  Pragmatic Project Automation: How to Build, Deploy, and Monitor Java Applications – The Pragmatic Starter Kit Volume 3.  Raleigh: Pragmatic Bookshelf, 2004.

Hunt, Andrew & Thomas, Dave.  Pragmatic Programmer, The: From Journeyman to Master.  New York: Addison-Wesley Professional, 1999.

Spolsky, Joel & Atwood, Jeff.  Stack Overflow Podcast, Episode 23 – Seven Mistakes Made in the Development of Stack Overflow Segment.

Beck, Kent.  Test-Driven Development by Example.  New York: Addison-Wesley, 2003.

Beck, Kent & Andres, Cynthia.  Extreme Programming Explained: Embrace Change.  New York: Addison-Wesley, 2004.

Hass, Kathleen B.  From Analyst to Leader: Elevating the Role of the Business Analyst – The Business Analyst Essential Library.   Vienna: Management Concepts, 2008.

Cockburn, Alistair.  Writing Effective Use Cases.  New York: Addison-Wesley, 2000.

Ruby Module Test::Unit.  http://ruby-doc.org/stdlib/libdoc/test/unit/rdoc/classes/Test/Unit.html

Vermeulen, Allan; Ambler, Scott W.; et. al.  Elements of Java Style, The.  New York: Cambridge University Press, 2000.

Baldwin, Kenneth; Gray, Andrew; Misfeldt, Trevor.  Elements of C# Style, The.  New York: Cambridge University Press, 2006.

Ambler, Scott W.  Elements of UML 2.0 Style, The.  New York: Cambridge University Press, 2005.

Knuth, Donald.  “Literate Programming” in Literate Programming, CSLI.  1992.

Wikipedia Contributors.  “Capability Maturity Model Integration”.  http://en.wikipedia.org/wiki/Capability_Maturity_Model_Integration November, 2008.

Wikipedia Contributors.  “Six Sigma”.  http://en.wikipedia.org/wiki/Six_Sigma  November, 2008.

Wikipedia Contributors.  “Lean Software Development”.  http://en.wikipedia.org/wiki/Lean_software_development  November, 2008.

Wikipedia Contributors.  “Risk Management”.  http://en.wikipedia.org/wiki/Risk_management  November, 2008.

Adams, Scott.  Dilbert Principle, The.  New York: Collins, 1997.

Poppendieck, Mary; Poppendieck, Tom.  “Why the Lean in Lean Six Sigma”. http://www.poppendieck.com/lean-six-sigma.htm  November, 2008.

Product References

OpenFire Instant Messaging Server
Spark Instant Messaging Client
Yahoo! Zimbra Email and Groupware Server
SSL-Explorer VPN Server
DokuWiki:
Microsoft SharePoint
Alfresco CMS
Vanilla Forum
phpBB Forum

In examining the R.M.S. Titanic from a perspective of project management it is important to first identify what type of product this project was set to produce. Unlike many projects where the final customer will own the final product the Titanic was designed to deliver a service, particularly a ferry service, to its end customers. This creates an interesting challenge in discussing “Project Titanic” since most views of project management see a project as having a discrete beginning and end as well as clear, well-defined stakeholders.

In the case of a project such as the R.M.S. Titanic we can take two views and approach the issue from two sides. In one case we have the project by which the three ships of the Olympic class were conceived, designed, built and delivered to White Star Lines. In the other case we have the R.M.S. Titanic as it was customized beyond the extend of its elder, the R.M.S. Olympic, completed in initial production and delivered, as a service, to the passengers which it was to ferry between Southampton and New York. To maintain scope I will not discuss the even larger project of testing, bug fixes, repairs, scope changes and enhancements that were applied to the two sister ships after the sinking of the R.M.S. Titanic. Both the R.M.S. Olympic and the H.M.H.S. Britanic saw many changes during their years of service including the re-scoping of the Britanic from the role of an ocean liner to that of His Majesty’s Navy’s principle hospital ship during World War 1 and the outfitting of the Olympic with a double hull and additional life boats as the crew refused to sail her until she had been made more safe. (“Olympic”)

It is my goal here to examine the Titanic as a service from conception to service delivery and, ultimately, service failure. From this perspective the Titanic can be treated much like one would treat a modern Software-as-a-Service (SaaS) project. Because of the nature of a ship such as the Titanic or SaaS products such as Salesforce.com or SugarCRM we need to consider the intended lifespan of the product and the ongoing upgrades and maintenance that will be needed to keep it operating. The Titanic requires a huge staff of pilots, seamen, chefs, porters, stewards and more while at sea and required re-outfitting, repairs and, had she survived, she would have needed a new double-hull like the R.M.S. Olympic received. A SaaS project will similarly require a staff to maintain the datacenter and networking, ongoing upgrades and bug-fixes, new features, etc. In both the case of the Titanic and of a SaaS project there is a real potential for a service disruption that could prove to be extremely costly. Maintaining steady, reliable operations is a major factor in the success of either of these business plans.

Let’s begin our analysis of the project to bring the Titanic to fruition by examining the stakeholders. We can easily identify the passengers and crew of the Titanic as stakeholders, White Star Lines as a company as well as the project engineers, Harland-Wolff as the constructors, Alexander Carlisle and Thomas Andrews as shipwrights and designers at Harland-Wolff, Captain Edward John Smith who was responsible for service delivery and finally White Star director Joseph Bruce Ismay and his executive staff who were in the role of project sponsor. In any project of this size there will be many important players all of whom have some stake in the project. We will focus just upon these key people as the most prominent stakeholders in the Titanic project.

The Titanic project most closely resembled a Waterfall Model in IT Project Management parlance. The process started with a high level concept coming jointly from Joseph Bruce Ismay representing White Star Lines and Lord James Pirrie representing their ship building partner, Harland-Wolff. The project was conceived jointly between the two companies. The Titanic would offer great prestige and profit potential for both firms and would require a large investment from both. While we do not appear to have the original project charter available to us today we can view the meeting between Ismay and Lord Pirrie as the unofficial project charter and the initiation of the project. (Sadur)

The technical design of the Olympic class ships was undertaken by Harland-Wolff head shipwright, Alexander Carlisle, after high level plans had been drawn up by Ismay and Lord Pirrie. Carlisle was the lead designer on the project from its inception until 1910 when he retired and turned over the lead design role to Thomas Andrews the managing director of Harland-Wolff (Brander 1998). Andrews would be responsible for the final stages of the Titanic’s design. The Olympic, launched in the fall of 1910, would have been most likely completely designed under the direction of Carlisle. Since the Titanic shared almost all of the infrastructure of the Olympic (hull design, compass placement, lifeboats, bridge, etc.) we can safely assume that Andrews’ contributions to the design of the Titanic were mostly aesthetic or, in software development terms, “interface” related. (Thinkquest)

Because of the construction-like nature of shipbuilding and especially with mammoth ships such as the Olympics the design process involves heavy upfront design with very limited feedback loops later on once the designers can physically inspect the ship. In software terms this is referred to as “Big Design Up Front” or BDUF. In software where changing requirements are common this is generally considered to be a very bad practice but in mechanical and structural engineering this is simply the only reasonable solution.

As work progressed on the Olympics several decisions were made regarding the core infrastructure design of the ships. This is especially dangerous as the methodology in place for this type of project was not designed to allow for changes of this nature once the plans were approved. A ship is designed as a holistic system with interdependent safety systems and a high degree of complexity. Unlike most software it is very difficult to rapid-prototype a ship to the degree of accuracy needed to ensure safety. Making key changes to safety systems would have required a full scale reworking of the specifications to be done correctly. But since changes were made because of cost savings, timeline issues and luxury appointments there was little holistic thinking put into the changes. (Kozak-Holland)

The original design and intent of the Olympics was that they would incorporate the very latest technologies for safety. After the initial design was complete business pressure from White Star Lines and especially J.B. Ismay was put onto the architects and engineers to remove safety features in favor of first class luxuries. The two most famous changes were, of course, the removal of the lifeboats so that the views from the cabins would not be unnecessarily obscured and the modification of several of the bulkheads to no longer seal and to rise just ten feet above the water level in order to allow for an expanded grand ballroom facility. As with IT projects, engineering decisions for core systems are generally beyond the ken of business executives. Allowing business side decisions to influence otherwise technical decisions is dangerous as the usual precautions and thought processes are bypassed and expertise is overlooked. In this case these were issues pertaining to the care and preservation of human life. In software we seldom have such an important task at hand but allowing business managers without understanding of key systems to be involved in their design can be exceedingly harmful even if the result is as benign as loss of business or greater cost of operations.

One of the most dramatic problems that arose from late changes to the Olympics’ designs were that the changes, each being small on their own, were most likely never taken together and examined as a whole in the same manner that the original ship design was considered. When the lifeboats were removed the engineers involved were thinking that the ship was designed to be a floating life raft and that the purpose of the life boats was simply to ferry passengers from a motionless ocean liner to another ship in the “worst case scenario” that the Titanic or Olympic were to lose power. Even a collision was expected to only make them float low in the water, not to sink. The life boats were removed until only the minimum legal number remained at the behest of White Star Line’s executive committee. To the engineers this would have been an acceptable, although not recommended, safety trade-off. The design of the ship was such that having additional life boats was not a legal requirement nor was there any driving need to keep them as the usefulness of the additional boats was believed to be minimal. In the end, it would not have been the decision of the engineers but of White Star who was the final customer to the shipbuilders and whose decisions provided them with their livelihoods.

On its own the decision to remove the life boats would, most likely, have been a minor one. But additionally the decision was made to change the key structural design of the Olympics from having all tall bulkheads to having four of them be significantly lowered to just ten feet above the waterline meant that the concept of the ship as being a floating life raft was compromised. The bulkheads were never truly water tight as the marketing materials would have lead us to believe but they were quite tall and very water “resistant” and would have most likely been able to keep water from traveling between them under even a very serious hull breach. As the initial design of the ship was replete with safety features this would have been considered, like the life boats, to be a redundant feature and its removal would have only been lowering the ship to more common safety criteria. Taken individually the changes were mostly innocuous, but taken together the changes became a complete redesign of the ship and completely disastrous. At no point did the qualified engineers go back through and do a complete reassessment of the safety features of the ship with all of the changes in place.

In some respects we could look at J.B. Ismay as being a micro-manager. He did not trust the decisions of those that he hired for the purpose of their technical expertise and overrode, either directly or through indirect pressures, their decisions. Micro-managing has many negative results. The obvious being untrained and unqualified managers influencing decisions that others believe to have come from qualified professionals. Others include eroding the value generated by the project team and degrading employee loyalty and morale.

In ship building, in situations where ships are built of a class such as the Titanic, we need to consider three principle project phases – design and construction of the prototype, the Olympic; design refinement and construction of the Titanic; and finally service delivery and operations. In the case of the Titanic in particular we see that the principle design and any structural changes were made before the completion of the Olympic. Thomas Andrews sailed on the Olympic but this was mostly so that he could make aesthetic modifications for the Titanic as it was far too late for structural work to be changed at that point. For the same reason, Andrews was sailing on the Titanic so that he could do similarly for the soon to be launched Britanic (known to Andrews as the Gigantic.)

In terms of project scope we can see that the project adhered closely to the initially established plan. Construction was done based on preapproved plans with little change. The most dramatic changes in the scope occurred during the construction of the Titanic when the project had to be halted in order to accommodate the repairs of the Olympic. Both parties, Harland-Wolff and White Star Lines, understood that the Titanic would be delayed but that the serviceability of the Olympic took precedence. A major factor in any type of capital construction project is the need for contract level agreements between build phases as scope change is nearly impossible to accommodate once construction has begun. (“Olympic”)

It is difficult to find software projects that closely follow this type of model with a production prototype followed by a series of production implementations based upon the prototype but not identical to it. The closest example of this that I can postulate is the enterprise resource planning (ERP) package SAP. With this package, and others of its ilk, customers buy the package based upon its prototypical performance and features and then either on their own or through a consulting firm or the original vendor will heavily modify the package to their own needs. Generally the advantage to such an approach is that the core of the software package, the infrastructure, is very stable and well tested and often used across a wide degree of situations giving it both project side as well as client side testing. Care must be taken, of course, because customer initiated modifications will not have the benefit of the deep testing that one hopes has been performed on the core infrastructure nor do the changes have the benefit of “many eyes” from the wider client community.

In the case of the Olympic class ships there was serious testing done on the fifteen foot model of the ship and testing was done on the Olympic upon completion. With a ship of the Olympic’s complexity it is critical that real world testing be performed in addition to unit testing of individual systems. The Olympic was put through the usual testing measures that were standard for a ship of this type. However, when the Titanic was built the builders and the cruise line decided that as the Titanic was essentially a copy of the Olympic that the testing done and the ongoing successful use of the Olympic was test enough for the Titanic and the Titanic received very little additional testing. This, however, is not a best practice as mariners know that all ships, even copies, behave differently and each vessel is unique and must be tested on its own. (Kozak-Holland)

The Titanic was given almost no time for testing or performance trials. This came about partially because the Olympic had had a serious accident and had to be taken to the shipyards in Belfast and repaired. While the Olympic was under repairs the Titanic had to wait as only one ship of that size could be worked on at one time. This placed business time constraints on the Titanic as she was scheduled for regular transatlantic route duty and was needed immediately. Because of this, some additional testing that would likely have occurred was skipped and the Titanic was sent out with its primary testing being the journey from Belfast to Southampton, and, even on this leg of its journey, there was at least one paying passenger making it more a low capacity true voyage than a proper test. (Kozak-Holland)

It would appear that White Star Lines and J. B. Ismay were quite willing to take on exceptional project risk in order to get the ship into regular service as quickly as possible. Through standard maritime procedure they mitigated much of their capital risk through maritime insurance. This would protect them against many potential unknowns.

During the last half of the nineteenth century it was becoming increasingly common for both shipping lines as well as governments to see risk as a low priority issue. The S.S. Great Eastern built in 1858 is considered to have been, and proved to be in real world cases, far safer than the designs of the increasingly unsafe ocean going vessels that followed her over the next fifty-four years. Conditions would continue to decline until companies and governments reevaluated the situation in the wake of the Titanic’s sinking. It is argued that shipping companies saw acceptable safety track records as justifying their lackadaisical attitude towards safety over decades of relatively incident-free shipping. Financial market pressures won out favoring companies with loose safety standards encouraging the industry as a whole to move away from expensive risk management. (Brander 1995)

Under the guise of further mitigating risks due to a lack of testing and training, several crew members, most notably Captain Smith, were brought over from the Olympus to sail the Titanic on her maiden Atlantic crossing. This could be seen as Ismay seeking experience which would appear to lessen the “unknowns” derived from sailing a ship without testing her directly but at least having the maximum experience from the prototype vessel. However, this may not be the underlying reason for the decision and it is very possible that Captain Smith was chosen because his position with White Star Lines was rather questionable after he had just recently caused a serious accident involving the H.M.S. Hawke which had caused the Olympic to need emergency repairs and had delayed the sailing of the Titanic. Captain Smith was likely nervous, concerned for his career and in little mind or position to act as the final level of responsibility aboard the ship if pressures from the firm directed him against his better judgment. This may have been exactly the operational loophole that White Star Lines was seeking. This situation was probably exacerbated by Captain Smith coming in too close or too fast to the S.S. City of New York, when moored in Southampton, causing it to break its moorings and nearly collide with the Titanic. (Kozak-Holland)

Under customary maritime law, a captain is the absolute commander of the ship and has complete jurisdiction while at sea even if higher ranking officials, military or civilian, are aboard. The captain has moral and legal responsibilities first to the lives and safety of the passengers and crew and then to the cargo and the ship. (Kuntz)

Once the Titanic was built, outfitted and available for sailing we see a stage change and we move into the service delivery phase of the overall Titanic project. In this stage we are past the traditional stages of project management. In most scenarios a client would now have taken possession of the finished product. But in the case of the Titanic this becomes the service delivery phase.

Under service delivery White Star Lines took responsibility for any new issues that would arise with the ship. At this point the traditional system of design – build – test would no longer be used and instead the service delivery would be overseen by standard operating procedure or SOP. Ongoing ship modifications, repairs, tuning and the like would still continue but these would be designed to not be at the level of requiring a service interruption but would be minor and done without the knowledge of the final customers – the passengers. It is at this stage that the passengers arrive as our most critical stakeholders because, in this scenario, they are not just financial stakeholders but are literally staking their very lives on the reliability of the ship and the operations of the crew.

In the Agile Project Management community there is a fable often used to denote roles within the stakeholders. These roles are known as the pigs and the chickens. The fable tells us of a chicken and a pig who are interested in opening a restaurant together. The pig asks the chicken what they will serve. The chicken replies, “Well, bacon and eggs, of course.” To this the pig responds “I don’t think that I am interested. You will be involved but I will be totally committed.” (Schwaber 7)

The pig and the chicken metaphor is normally used to express the difference between stakeholders who have real money or careers on the line versus stakeholders who have a vested but non-critical interest in the project. The chickens would prefer not to see a project fail but failure is not necessarily devastating for them. In the case of the Titanic we see that the financial stakeholders, Harland-Wolff and White Star Lines, were effectively chickens. They had much to lose but their investment was insured and later, we will see, the government was even willing to protect companies of this nature at this time due to the pending war with Austria and Germany. Neither White Star nor Harland-Wolff were “totally committed” – they had a definite interest and the success of the Titanic was extremely important to them but the passengers and crew of the Titanic were truly the pigs here willing to put their very lives at stake. Seldom is the chicken and pig metaphor more appropriate.

In order to ensure a higher quality of ongoing service a guarantee group from Harland-Wolff was present on the maiden voyage. This team included many important members of the Harland-Wolff design and construction staff including the lead designer Thomas Andrews. This guarantee group was customary on all major projects of Harland-Wolff. This staff would use the voyage time to assess the construction under differing conditions from their tests, gauge customer satisfaction and look for opportunities for improvement. Thomas Andrews had already sailed on the Olympic for this very same purpose and had made many tiny changes to improve the Titanic. He would spend part of this journey, for example, designing less expensive clothing hooks for passenger rooms. (“Guarantee Group”)

The Guarantee Group comprised specialists from many different technical practices within Harland-Wolff. We see representation from the fitters, electricians, joiners, draughtsmen, design team and more. This group, with their varying specialist areas and their differing levels of experience with both seniors and apprentices included, would have been an exceptional cross-section of the project team that built the ship. Their presence aboard with the care given to appraising the workmanship, design and other final components can be seen in two principle ways.

In the first way we can see this as a “port-mortem” performed on the Titanic Project. It was the role of this team to assess the technical success of the project and to look for areas for improvement as well as to generate “lessons learned” so that future projects could benefit from the experience gained here. Considering the cost of the transatlantic trip and the time spent away from their regular duties this was a serious investment in project knowledge by Harland-Wolff and extremely commendable.

Taken in another light, this guarantee group could be seen as providing feedback on a construction iteration. The Olympic’s construction being the first iteration, the Titanic’s being the second and the Britanic’s being the third. In this approach we see a type of Agile feedback loop being utilized, as much as possible, to allow for customer input even on such an extreme capital construction project. The iterations are very long, but this is by necessity. In this way we can view the Titanic as both a project unto itself, being a discrete deliverable, as well as being part of the ongoing project to deliver passenger service via the Olympic family of vessels.

The guarantee group being aboard ship would have presented the opportunity for the technical teams to get a first-hand appreciation of the real world application of their product. Rarely would a technical specialist be in a position, in 1912, to travel on a ship of this level of luxury. Without the sponsorship of Harland-Wolff in providing this chance for its staff to witness their own workmanship at work they might never understand their own roles in providing services to their final customers.

Having apprentices included in the guarantee group meant that direct one-on-one or small group, informal training could be performed. The apprentices and the senior technicians would have had many days to work together, and the apprentices would have had a great opportunity to learn from their seniors in a setting conducive to team building and knowledge transfer. In many ways we can see this time as being similar to off-campus team building sessions or retreats popular with many companies and project groups today.

Where we find the biggest surprise in our analysis of the Titanic is the almost total lack of standard operating procedure in use on board the ship. Some processes and procedures were documented but many were not. Examples of processes that were not standardized included key communications processes such as moving messages from the wireless office to the bridge, alerting passengers to the ship sinking and alerting the bridge that the crow’s nest had seen an iceberg. (Kozak-Holland)

Standard Operating Procedures are absolutely critical in any service delivery situation. In some companies these can even be considered so valuable as to be the core intellectual property of the company. Without the SOP a company is no more cohesive than the inherent “teamness” of the staff which, in cases of new employees, may be nominal. Staff will have to rely totally upon best practices, convention, informal staff instruction or, hopefully, training to learn their jobs and processes. But these will not be standardized if they are not written down and training will inevitably vary from trainer to trainer and no employee can retain all instructions for all possible scenarios.

Under normal conditions the lack of standard operating procedures may be of relatively minor importance. Staff can perform most job functions adequately, especially if trained, without needing an SOP. If they did they would need to carry the SOP with them at all times. When the SOP becomes extremely critical is when “normal operating procedures” are no longer available or, in more modern terms, when operations is no longer under BAU (Business as Usual) conditions. For the Titanic, BAU conditions were broken several hours before the iceberg incident.

In the case of the Titanic it is difficult to discuss standard operating procedures without also discussing communications. So we will begin with communications under BAU conditions and then see how the lack of an SOP caused the situation to deteriorate rapidly.

The Titanic was plagued with communication design flaws from the beginning. The wireless room, responsible for all communications going in and out of the Titanic, was not operated by White Star Lines but was instead staffed by Marconi personnel who were paid to communicate first and foremost for the passengers and for the ship only if time allowed. The wireless operators were overworked and undervalued and would often not relay messages to the bridge because they had other duties that were considered to be of a higher priority by Marconi, their employer. At least eight iceberg warnings were sent to the Titanic’s wireless room but only some of these were relayed to the bridge. (Kozak-Holland)

In this case it is important to emphasize the importance of managing third party contractors via a service level agreement. White Star Lines, in allowing the Marconi Company to staff its wireless room, should have had a clear SLA demanding that safety and emergency communications for the ship take absolute priority over the personal messages of the crew. They should also have not allowed Marconi to make an additional profit or be financially benefited by not following the SLA. As an outside contractor Marconi should have had a contract that was designed for mutual benefit – that is that when executed as stated that it was to all parties mutual benefits to act correctly. Contracts between vendors that give financial incentives for a vendor to work against the good of their client are very unwise.

The most important single incident involving the Marconi operators was the final iceberg warning sent by the SS California, which was extremely near to the Titanic – so near that later it would be the ship to see the emergency rockets from the Titanic. The California radioed the Titanic a warning that it had become trapped in pack ice and could not proceed, at any speed, due to the dangerous conditions. The Marconi operator replied “Shut up, shut up, I am busy. I am working Cape Race and you are jamming me.” It could hardly be made more clear where the priorities of the wireless room lay even with such danger looming so closely. Not only did the wireless room not keep communications open with the California but they also declined to inform the bridge of this final external warning. In frustration the California’s radio operator gave up his attempts to warn the Titanic, turned off his wireless and went to bed. The Marconi operators not only failed to heed the warnings but alienated external channels such that the only ship near enough to rescue them did not respond when the Titanic began to sink. (Kuntz)

Communications from the bridge to the crew at large and the passengers had no official process, was manual and was, at best, done in a best effort manner if it was attempted at all. The bridge did not notify anyone that a collision was immanent and no one was braced for what could have been a very serious impact. Once the ship began sinking it took over an hour before the bridge began informing the rest of the ship that they were going down. Key information affecting the lives of the passengers and crew was kept from them and held by just a few bridge personnel who were most likely hoping to keep the incident a secret or to minimize the publicity to the obvious risk to human life. As there was no system or process for communicating from the bridge to the ship in general this was a simple matter as it took a concerted effort to inform the passengers of any news at all. (Kozak-Holland)

Communications amongst the crew were little better. The Officer of the Watch, for example, was located outside of the bridge but his critical communications links were located inside of the bridge house. So the watch was unable to rapidly communicate to any other bridge personnel or to coordinate with the crow’s nest and other related functional areas. The crow’s nest and the watch were connected by a one way bell system that did not allow them to communicate in duplex and was very slow, and the watch had no means of feedback from the quartermaster at the helm when an emergency command was given. Commands were given by shouting from the open air towards the enclosed bridge. The watch could only hope that the quartermaster inside the bridge had heard, understood and was acting upon those commands. Communications from the standard compass were just as bad. The compass, instead of being located above or near the bridge, was placed far to the aft and the bridge was forced to coordinate with the compass on a regular basis which caused much confusion and delays. Little, if any, thought was put into making the bridge effective or safe. This lack of design for communication certainly did little to help the Titanic when rapid and accurate communications were necessary. (Brown)

When external communications to the White Star office in Boston were finally sent the information relayed was that there was no serious damage and that the incident was very minor. Unlike point to point information that is common today, however, this information was broadcast and could easily be intercepted by other ships and relay stations. Ship-to-shore communications were often used to effectively release information to the press under the guise of an internal communiqué. So instead of relaying honest and critical information the wireless was used as a marketing tool. What was sent was not a distress signal but was effectively nothing more than a press release designed to put “spin” on the situation. (Kozak-Holland)

Communications are key at any stage of any project. In the case of the Titanic the catastrophic situation highlights issues that occurred because of communications, but this is simply a worst case scenario. Projects need to have as much up-to-date and accurate data as possible when making decisions. Without it decisions are made using only a partial available picture and the less correct information available the less likely that a good decision can be made.

Perhaps the greatest project management issue affecting the Titanic, however, was its lack of standard operating procedures. The SOP should have been produced as an essential project deliverable during the later stages of the construction process. For a ship to have been deemed seaworthy when there was no SOP to operate her is truly inconceivable. Even the most agile of development methodologies fails to overlook the need for end-user documentation.

Since the design and construction portions of the project had failed to provide the crew of the Titanic with an SOP or, at least, a reasonable SOP (there were some generic standard procedures in the White Star Lines manual itself) there were no clearly defined rules or processes for dealing with communications, tracking alerts, providing warnings, etc. There was no emergency procedure to be followed and so the crew was forced to act on nothing other than experience and general mariner knowledge.

It is at this point, when examining the actions of the crew under emergency conditions, that we witness the complete breakdown of the command structure. In a traditional business the business executives are generally accepted as having the final decision-making power on any corporate action as long as it falls within legal boundaries, and often when it does not. In the average business a bad operational decision results in loss of revenue not the loss of life. A wise executive will understand the need to not override the decisions of those specialists hired to make operations decisions or possibly a board will require that an executive listen to her staff. Nonetheless, the idea of business side executives interfering with project operations is against best practices and is widely accepted as being a bad course of action.

In the case of the Titanic, Captain Smith was in command of the vessel at sea and was personally responsible for the ship and the souls aboard. His boss, J.B. Ismay, may have had the ability to have Smith removed from command upon returning to port but at sea he did not nor under British mariner law did he have the right to make commands from the bridge as he was not a licensed mariner. (Kuntz)

During the time leading up to the iceberg collision J. B. Ismay had been pressuring Captain Smith to drive the ship at an irresponsible speed – in excess of twenty-four knots or seventy-five revolutions. The Olympic, being considered the “test” for the Titanic, had never crossed the Atlantic at this speed and the Titanic was now operating even outside of the range of tests performed on the Olympic without even time enough to have performed a single Atlantic crossing under normal conditions. Ismay and Smith drove the Titanic beyond its known performance parameters and, more importantly, beyond the crew’s known operational parameters. It was simply unknown what operational risks would be involved with the ship at this speed. To maintain what should have been considered an unsafe speed while also going into waters known to be strewn with icebergs was extremely foolish.

Whether because of panic, confusion, insecurity, etc. we do not know but when the Titanic struck the iceberg Captain Smith allowed a layman, J.B. Ismay, to come onto the bridge and to begin making executive orders as the acting ship’s master for which Captain Smith had the right and obligation to have Ismay removed. Ismay made key operational decisions including emergency communications, passenger notification and, most importantly, to move the Titanic forward off of the ice shelf which is believed to be the actual cause of the ship’s main rupture and then to continue the ship moving forward at slow speed pulling the hull apart even after additional information was available that the ship was going to sink. (Kozak-Holland)

Given the distance in time that we are now from the Titanic it can be difficult to assess processes followed and to know what went right with the project when we know so much that went wrong. The sinking of the Titanic is so iconic in our minds that to see it as anything but a marketing and organizational disaster is difficult at best.

In the end the Titanic project was immense but well managed. Scope was controlled and changes accommodated when necessary. Large design up front with a well established contractual interface to the construction phase was used and this cementation of the specifications allowed for careful and accurate scheduling. The processes by which the ships were built were standard and well known. Using historical construction data Harland-Wolff was able to accurately predict the time needed for construction allowing White Star Lines to begin marketing and sales long in advance of the actual sailing of the ships. The Titanic, being an almost identical copy of the Olympic, had even fewer surprises. The only true surprise resulted from the change of priorities from White Star Lines that resulted in the Titanic project being put on hold for approximately one month.

In the words of J. Bruce Ismay “She [Titanic] was not built by contract. She was simply built on a commission.” This indicates that exceptional authority was granted to Harland-Wolff to use their own processes and oversight to ensure the delivery of the Titanic. The two companies operated more as partners than in a vendor-customer relationship. (Kuntz)

Project risk for the Titanic was handled poorly relying heavily upon external insurers and, in the end, the British government to protect the company from liability lawsuits at the expense of the primarily British and American passengers. Risk was considered to be very low and because of this many careless decisions were made first with the Olympic and then, when operational disasters were minimal, even moreso with the Titanic. Careful risk assessment was not made. Expert mariners could easily and quickly have defined many risk areas that needed to be addressed. Issues such as the lack of a complete Standard Operating Procedure would have been flagged and could have easily been handled since resources for this would not have needed to have come from the current Titanic team and would not have impacted the delivery date or any of the variables that we understand now to have been of primary concern to White Star Lines.

Communication on the project seems to have been handled very well until service delivery began. At this point design flaws, questionable decisions and the lack of SOP came to bear on the communication network on board the ship. This communication could be considered to be operational and not project based but the argument is semantic. The issues with the Titanic were holistic and with proper design methodologies being followed risk analysis would not have been missed which would have forced the creation of the SOP which would have highlighted or perchance even fixed the communications issues.

At its core the question was one of quality. The Titanic was proposed and sold as the highest quality transatlantic travel option. Quality was heralded, directly or indirectly, in almost every breath spoken about the Titanic. The customer interface was kept as clean and concise as possible. No expense was spared if the results would be witnessed by a customer. But the underlying core or infrastructure of the project (non-functional requirements according to Kozak-Holland – although I do not agree with their use in this context) on which this “quality” was to rest was ignored and the true quality of the Titanic and the quality of the operations of White Star Lines was to become ultimately evident.

Bibliography and Sources Cited:

Schwaber, Ken. Agile Project Management with Scrum. Redmond: Microsoft Press, 2003.

Kuntz, Tom. Titanic Disaster Hearings: The Official Transcripts of the 1912 Senate Investigation, The. New York: Pocket Books, 1998. Audio Edition via Audible.

Kozak-Holland, Mark. “IT Project Lessons from Titanic.” Gantthead.com the Online Community for IT Project Managers. (2003): 22 February 2008

Brown, David G. “Titanic.” Professional Mariner: The Journal of the Maritime Industry. (2005): 23 February 2008

Sadur, James E. Home page. “Jim’s Titanic Website: Titanic History Timeline.” (2005): 23 February 2008.

ThinkQuest Library. “Designing the Titanic.” (Date Unknown): 25 February 2008.

Titanic-Titanic. “Olympic.” (Date Unknown): 25 February 2008.

Titanic-Titanic. “Guarantee Group.” (Date Unknown): 25 February 2008.

Brander, Roy. P. Eng. “The RMS Titanic and its Times: When Accountants Ruled the Waves – 69th Shock & Vibration Symposium, Elias Kline Memorial Lecture”. (1998): 25 February 2008.

Brander, Roy. P. Eng. “The Titanic Disaster: An Enduring Example of Money Management vs. Risk Management.” (1995): 25 February 2008.

Additional Notes:

Mark Kozak-Holland republished his 2003 Gantthead articles on the Titanic into a book:

Kozak-Holland, Mark. Lessons from History: Titanic Lessons for IT Projects. Toronto: Multi-Media Publications, 2005.

More Reading:

Kozak-Holland, Mark. Avoiding Project Disaster: Titanic Lessons for IT Executives. Toronto: Multi-Media Publications, 2006.

Kozak-Holland, Mark. On-line, On-time, On-budget: Titanic Lessons for the e-Business Executive. IBM Press, 2002.

US Senate and British Official Hearing and Inquiry Transcripts from 1912 at the Titanic Inquiry Project.

Non-Measurable Organizational Value is the concept of corporate benefit being derived from a project that is inherently non-measurable. To some degree all value is difficult if not impossible to measure accurately except in extreme circumstance. Organizations are complex entities and any project or process is just one of many projects or processes that all contribute in some way to value.

Let us look at a quick example of a difficult to measure project. Project X at ABC Corporation (they give all of their projects letter-names for some reason) is a project to add four important new features to one of the company’s software products. The project runs its course and is estimated to have cost the company almost exactly $100K to complete. The new features are now available in the product which is available to existing customers as a free upgrade and are included in the shipping product to all new customers. Now we must determine the Organizational Value or OV of these new features.

These new features are designed to derive their organizational value from several planned sources. The first source is through “feature marketing“. In this capacity we are left attempting to derive, generally, through collected sales data both before and after project completion what the change in sales is because of the new, added feature. This has to be balanced against any changes in sales or marketing, changing market pressures, increased product maturity, changes in competing products, etc. Even asking customers to voluntarily report on this data can be very misleading in the best of circumstances.

The second source of derived organizational value is through customer retention. This is far more difficult to calculate than the somewhat ephemeral marketing and sales aspect of the first source. To some degree you can calculate the number or percentage of customers who decide to go through the upgrade process if they are downloading the update from ABC’s web site. But you are still left estimating what percentage of customers did the upgrade simply because they felt that they should use the latest version, who wanted to get the features but would not have switched products to get them and which customers would actually have taken the effort to switch to another product to acquire those features.

This approach does not consider the market affects that ABC’s new features have had on its competitors. In some cases by supplying in-demand new features ABC may have driven the market forward forcing competitors to include those same features, but it may also have done research and development that now its competitors can copy at lower cost. Or perhaps it has implemented features so costly or specialized in nature that competing products avoid implementing those same features simply because they are already available on the market. Estimating ABC’s market effect is very difficult if not impossible.

The third source of revenue is difficult to calculate as well. This third source is through market preparation. By this I mean that two of these four features implemented by ABC Company are building blocks that are expected to lead into another project that will provide a number of really incredible and difficult to produce features in another year. A certain amount of planning and framework design was done during this project in order to prepare for the following project. A few hundred hours of manpower were put into this robust framework that would have been excessive for just the current features but will allow future features to be added in more easily. This technology investment will not see dividends until other projects, seeking their own organizational value calculations, are layered on top of it.

In this example of ABC’s Project X we see that this project was based mostly upon Non-Measurable Organizational Value. Some of these value sources could be examined and an estimate of OV could be extracted. But this is, at best, guesstimation and the organization would need a very carefully managed process to keep this type of guesstimating consistent and fair between projects. But there are other NMOV concepts that we should also address.

One type of NMOV is employee morale and the related “development velocity.” Employees, especially technical workers, are highly affected by their work environments and generally desire to be allowed to produce good products. Developers and analysts will often state that the value and quality of their work is a driving factor in their level of job and career satisfaction. A company with high morale, satisfaction and happiness will generally get better products and it will get them out the door faster. Teams are more likely to gel. Sick time goes down. Communications go up. A project will not often be undertaken purely out of concern for employee morale but a feature, a methodology, a technology or a technique might be chosen for just this reason. A project might be written in Java instead of C++ just to keep a team more interested in their work, for example.

Now we can attempt to measure employee morale in several ways but we have two main arguments preventing us from doing any meaningful measurement. The first is that there is no good means of measuring an increase in morale based on project decisions. Finding single points of morale boost or morale decrease may be possible but determining overall morale deltas caused by a specific project separate from the overall organization culture is simply unreasonable. The second argument, against being able to measure the OV, is that even once morale change is estimated it is then not possible to determine the degree of effect that this will have against the short term or long term organizational bottom line. And, of course, we must consider the morale bolstering affect of an organization being willing to sponsor projects for the purpose of or, at least, with consideration towards employee happiness.

But this is not to say that employee morale has no value. Certainly it does. I believe that it is an important factor in organizational health. A very important factor indeed.

Any project looking to determine its own value cannot truly do so without considering the implications of non-measurable organizational value. But to what extent should these factors be considered? To this, I believe, there is no simply answer. In an extremely large enterprise where many years of metrics could be collected and careful research into marketing, market pressures, competition, morale, productive velocity and more can be collected and calculated, over a very large system of hundreds of thousands of employees, I believe that we should be able to see reasonably consistent trends that will lend themselves to reducing NMOV factors into estimable organizational value. However even in this circumstance these calculations will have to be done at a very high level of abstraction with a significant amount of organizational research ongoing at all times. And results, unlike OV from specific projects, are most likely to only have a good degree of confidence when undertaken as organizational directives and initiatives and not on a project by project basis.

Large enterprise organizations, those with tens or hundreds of thousands of employees, are often caught by their own bulk and momentum and find that many NMOV factors do not exist for them in the same manner than they exist for small companies. At the one extreme a behemoth manufacturing company with a half million employees will find enacting organization wide morale or velocity initiatives to be difficult to implement, difficult to measure and that the effects are minimal as the momentum of the firm keeps these changes from trickling down to the majority of the workforce without being watered down by the established corporate hierarchy until even the attempt at change is seen as a waste of effort. Large enterprises often settle into a relatively stable state of morale with only truly significant events having a serious or long lasting effect.

Small companies, especially those under five hundred employees, can find a much higher value in NMOV, in my opinion. Small companies gain some of their greatest advantages through the leverage of NMOV. In a tiny company of twenty people one great project could invigorate the entire company for potentially years. Culture and attitudes can change almost instantly and truly great velocities can be achieved and maintained. Smaller companies need to be more attuned to the NMOV factor than their large counterparts. It can be both an advantage and a disadvantage. Small companies do not have the luxury of the employee scale buffer (read: organizational momentum) to keep morale busting events from dragging a company down quickly.

Some larger companies have become famous for managing to keep morale and culture at the forefront of their priority list with exceptional results. Notably Microsoft and Google have become poster-children for great corporate culture even in large firms. And both have become well known for consistently delivering strong technical products, moving the market forward, breaking new ground and keeping employees very happy. Both are also known for investing heavily in research and development which are often incorrectly thought to be NMOV activities but, over time, R&D activities have a reasonable level of estimable value.

NMOV cannot be the sole driver of organizational projects but it should not be discounted. NMOV should be considered even if, at the very least, only from the aspect of attempting to mitigate negative NMOV behaviour or project choices. To some extent, though, I believe that NMOV should be estimated through soft calculations and guesstimation to be accounted for within organizational project portfolio planning as well as corporate culture fostering activities.

Perhaps NMOV, given the potential value that it can add to an organization, should be considered not to be non-measurable but to be immeasurable.

Marchewka further breaks down the term MOV and says that it must implicitly include the following: be measurable, provide value to the organization, be agreed upon and be verifiable. Let’s look at each of these.

Measurability is obvious yet extremely difficult. Many benefits of IT projects are “soft” and inherently unmeasurable. For example, a project that makes employees more happy cannot be measured as it is never possible to determine how much happiness is the result of any one project and how much other organizational efficiencies can be attributed to employee happiness and morale. And yet we almost all agree that happy employees work better, are more loyal, cost less and interact better with each other.

The idea behind measurability is that no project decisions should be made without a consideration towards how they will affect the project’s MOV. If a new feature is being considered, for example, then that feature should be compared against the MOV. If the feature will not increase the MOV then it should not be included. A relatively straightforward concept, but it basically states that only measurable value should be considered. This is not always intuitive.

Project Management by MOV should provide value to the organization. This is the underpinning of the MOV concept and is analogous to the concept of ROI. ROI, however, is a measurement of the difference between expenditure and the expect value to the organization. MOV does not seem to take into account the cost of its own provisioning and only looks and the measurable business value after project completion while ROI takes into account the cost of providing the MOV as well as having the potential to consider the non-measurable organizational value which may be the driving force or a project.

A project MOV must be agreed upon. Marchewka states that all project stakeholders should agree upon the MOV of a project before the project starts. This requirement includes making business stakeholders as well as technology stakeholders, such as analysts and developers, agree to the MOV before a project begins as a later measurement of project success. This is a difficult task as it is in one group of stakeholders’ interest to make the MOV high while it is in the interest of the technology stakeholders to make it low. This is especially difficult as it benefits the business side to trick or take advantage of the lack of business acumen from the technology side and requires the technologist to allow themselves to be judges of something that they neither understand nor ultimately control.

Verifiability of the MOV is key. Since the project’s MOV is measurable by definition it must then be verifiable. After the project has been completed the MOV is to be verified to determine if the project was successful or if it was not. However, Marchewka does not seem to address the issues of ongoing organizational value. A typical IT project will deliver negative value up front and will increase in value over time and then, eventually, decrease in value until it is replaced. True MOV would not be verifiable until the end of its lifespan.

For example, since code from IBM’s System/360 project from 1964 is still widely in use one would assume that IBM has not yet been able to determine the final MOV for that project. If their initial estimates had been extremely accurate and had taken into account a lifespan that might even top fifty years then the MOV would not yet be able to be verified as having quite reached its full value. Therefore a useful MOV is one that takes into account an acceptable lifespan of measurement but this introduces many more factors.

The categories here are separated by duty and represent the basic building blocks of the IT professions and disciplines. By no means is this meant to be representative of all job roles and career paths available to an aspiring IT professional or hobbyist but to provide some structure to make target careers less ephemeral. In the real world few, if any, IT professionals do the work of only a single job role without venturing outside its strict boundaries if such boundaries can even be argued to exist. In extremely large IT departments (those over 10,000 IT professionals) will often stick very strictly to descriptions such as these but small departments (say of only 10 IT professionals) may lump almost all skills into just two or three overarching job descriptions.

Programming: Of all professional areas within Information Technology the area of programming is surely the most well known to people outside of the profession. Programmers can work in numerous different technology areas, specialize in many different ways and can work with many different languages and platforms. Programming is often the area of IT that draws people into the field. Programming, more than any other IT discipline, is easy for people to begin learning early and is very accessible.

Programming, or coding, involves the writing of computer programs which can range quite significantly and job titles vary dramatically as the job descriptions begin to differ. Beginning professionals on the programming path are often just termed “programmers” and can expect to do programming projects that involve tiny pieces of larger systems. Programmers are almost always working on teams of programmers but can potentially live very solitary existences if such is desired. Programming professions allow for a very wide array of working environments. Programmers are more well suited to working flexible hours and from remote locations or “work from home” due to the nature of many programming projects.

As programmers progress along their career paths they can move up to positions like software developer, software engineer and software architect. Specialization within the programming realm can include system programming (working close to the hardware – highly technical), user interface programming (working closer to the end user experience – generally less technical and more creative and involved in the “human element”), database programmer, web application programmer, etc. Programming fields lend themselves to crossing into software design and management roles as well.

Systems Analysis and Design: Programmers may write software but systems analysts design it. Often the two roles are combined in what is called a “programmer analyst” as the roles are so closely identified. A systems analyst’s role is to define requirements and high level design for an application or program. Programmers are responsible with the low level design. A good analyst will have a very good understanding of programming, developer’s tools, architecture and more. It is a broad discipline that often involves a lot of customer or client interaction and the ability to translate requirements from clients outside of the IT field into useful requirements for design and for the programmers.

Systems Analysis is almost a management discipline and analysts will often cross that boundary many times during their careers. It is an exceedingly creative part of the IT field requiring a lot of critical thinking and “outside the box” contemplation skills.

Project Management: Any area of IT can have project management involved with it but this almost always applies to software project management or system project management. IT departments that include any number or programmers and possibly analysts will logically be charged with developing software. Project Managers oversee this process. Technical project management is generally closer to being a management discipline than an IT discipline but many PMs are highly technical and come from the core IT ranks as IT project management is so varied and different from project management in other areas such as engineering.

Hardware Support: Hardware support comes in two basic flavours – desktop hardware (which includes laptops and other commodity end user items) and server or datacenter support. Hardware techs range from consumer desktop support personnel that you will encounter at stores like CompUSA and BestBuy to server technicians working in the datacenters working with multi-million dollar hardware. the range is rather broad. Because desktop hardware has become so commonly known the “computer store” techs are generally not considered to be IT professionals any more than a car salesman would be considered a mechanic or a car designer. Sometimes a store tech job can provide leverage into the field but generally this is not the case. The technologies that are used in consumer PCs is enough different from enterprise business systems that the skills are generally not useful across the divide.

Some large companies maintain a staff of hardware technicians who work on desktop and laptop level hardware. Desktop class technicians are so identified with the CompTIA A+ certification that often times these job roles may be termed “A+ Techs”. This is generally a path towards the server technician positions. Server technicians need to be familiar with much more complicated and varied hardware and often work in large datacenters where there is little or no direct customer interactions. While desktop techs often interact to some degree with end users and desktop support technicians, server technicians generally interface only with systems administrators.

Networking: One of the core skill areas in IT is networking and communications. Networking is a relatively new discipline within the industry as computers used to exist primarily as stand-alone devices whether in homes or in business. But over the last few decades the idea of computers that are not a part of a larger network has gone from commonplace to practically unthinkable. Today even the most basic home computer is purchased to be an Internet connection node and not for the innate capabilities of the computer itself. Because of this networking has exploded into a very large, core discipline needing many qualified professional to fill out its ranks. Networking jobs generally fall into a few basic categories: network technician, network administrator and network engineer.

As you can guess from the job role names a technician’s general role is to deal with mostly “field” networking issues which often involves a lot of leg work, is more likely than other positions to place you in a remote office and often involves working with smaller categories of networking equipment but it is a stepping stone to high level networking positions. The network administrator is the position responsible for managing and running the day to day operations of the corporate network. Generally the network administrator is the last word in the company’s network operations. This can be a very senior position and while the job titles are few the discipline’s long term career growth is solid. A network engineer’s job would be to design a network. Often administrators and engineers are the same people but in large companies these roles are separated with engineers generally having a broader knowledge of network solutions and vendors and administrators having a more thorough knowledge of low level tuning and configuration of the equipment used at that time.

Systems: Possibly the largest of all IT disciplines is that of systems. The concept of systems is so large that it is difficult to define in any meaningful way and is often conceptualized as several sub-disciplines to make it easier to quantify. The basics are that the “systems discipline” involves any basic management of computer “systems”. This can mean management of end-user resources like desktops, laptops, PDAs, etc. as well as shared resources like servers. Generally a “systems” professional will work primarily with the computer’s operating system but this qualification is hazy at best. Any real work systems professional will have much overlap with other areas but core functionalities are generally more well defined.

Desktop and Deskside Support: The most common sub-discipline within systems is desktop support. This role is very difficult to separate from that of “Helpdesk” although the later is less of a distinct discipline but more of a delivery method of support. Most businesses separate helpdesk into a unified function that crosses many discipline boundaries.

Desktop Support involves the direct management of personal computers whether they are Windows, Mac, Linux, etc. A desktop technician will often work either directly with an end-users workstation or remotely via remote control technologies to help keep workstation resources working correctly, adding new software, etc. Desktop administration often deals with large numbers of desktop resources and generally handles password and account issues, large scale desktop changes, migrations, etc.

While desktop job descriptions are generally rather lean the field is actually extremely large. Almost every business requires a regular host of support personnel to keep the non-IT staff working on a day to day basis and will additionally utilize contract desktop support staff to augment internal resources as often “project” work will require far more people than a company can normally keep on staff. Many IT professionals who intend to enter almost any of the other disciplines will start their careers in the desktop support realm as it has the lowest “barrier to entry” into the field. But don’t be fooled. Just because it is easy to get into the beginning desktop support ranks there are many long term career opportunities within this field as well. Many professionals have long and rewarding careers without even leaving the desktop support arena.

In some large organizations there might even be a dedicated desktop engineering role specifically for those function of designing the operating system and application profile for corporate desktops. This position is almost always included in other job roles but can, potentially, exist on its own.

Server Administration and Engineering: The most visible and well known career path under the systems umbrella is that of server administration and its nature sibling, server engineering. These roles are so common that almost all businesses simply refer to them as system administration and system engineering.

Server support roles are involved with the designing, building (from a software perspective,) securing, deploying and managing the server resources of an organization. These servers come in a wide variety of types from Windows, Linux and Netware operating systems to application, database, web and email functions. Server support is a very large job role category that often spans entire careers from intern to retirement. This is one of the largest senior level career categories and is often a “target” career for people entering the IT field.

Pure server support roles as can be found in very large companies may be very strictly limited to supporting just the operating system and core functionality of a server. More often server system administrators will be involved in the running of extended functionality such as email, web, database and other software that is tied to the server.

Application Support: In large organizations when the system support role is strictly limited to the server’s operating system you will find dedicated application support personnel who generally specialize in a single application (such as Microsoft Exchange) or in a category of applications (such as Email) or in a suite of applications (such as Microsoft BackOffice including applications like Exchange, SharePoint, LiveCommunications Server, Project Server, etc.) More often you will find mixed server and application specialist who specialize on a particular platform and application combination such as iPlanet on Solaris or Apache on Linux, etc. Management Information Systems applications such as Enterprise Resource Planning (ERP) or Customer Resource Management (CRM) are common dedicated application areas as well.

Many companies have myriad internal applications that have been developed or customized either in house or through a consulting agreement and are considered to be a competitive advantage for the organization. These unique applications often require support as would any commercially purchased off the shelf application. In addition to the obvious role of application administration the role of application support is also common in large corporate entities. This is often called “operations” as this role functions almost as an organizational nerve center.

Database Administration: Known as a DBA, a database administrator is a special category of application administrator that is dedicated to the database technologies. Databases (such as Microsoft SQL Server, Oracle, Sybase, MySQL, PostgreSQL, etc.) are such a critical, popular, important and unique application that the field is considered to be its own area. There are many skills unique to the DBA profession that are not used or not widely used outside of database administration.

Database Designer: In a role somewhat related to both systems analysis and programming is that of database designer. A database designer’s job role is to work with application designers and analysts to design the database portion of an application. Databases are extremely complex types of software that generally require careful management and tuning and individual databases require detailed design which can be a significant portion of the design of an application.

Web Designer: Unlike the web application developer which is a popular programming job role a web designer fulfills the very popular function of designing web pages themselves. This is often considered to be a fringe IT job role because it is equally related to publishing, marketing and other, non-IT disciplines but because a truly qualified web designer needs a to be very skilled technically it is, in my opinion, a true IT discipline. Web designers get probably more opportunity for artistic creativity than any other IT activity. Often web designers will slowly more into programming to enhance their skill sets and will begin to become user interface specialized web application developers. But the leap from non-programming web design to web application development is a large one not to be undertaken casually. It is truly a change of discipline but between two disciplines that are closely tied together. Web design is by far the most prominent IT discipline to make use of traditional artistic abilities.

Security: While almost every job role needs to make security a part of their own discipline the enterprise has a place for overarching security personnel as well. IT, because of its ties to the company’s most valuable non-people assets – data, is integrally tied to security. The role permeates the field and is broad in its implications. Security professionals must be aware of everything from physical security, system security, network security, database security, programming methods, etc. In today’s IT professional realm security has become an extremely hot topic and it is very likely to remain so indefinitely.

Help Desk: This task is often placed in its own category because of the nature of the position. Help desk generally refers to the job role of the technical support call center. Help desk roles generally range from customer application support to remote desktop support. A help desk and an operations center will often be paired together or combined into one entity. Using current remote desktop management technologies such as RDP the modern helpdesk has taken on many of the job functions previously covered by deskside support. As networks become more stable and power powerful and as desktop management becomes more ubiquitous and far reaching the abilities for the help desk to cover most day to day support functions increases. Often the helpdesk is used as an aggregation resource to provide a single point of contact for any needs originating from an non-IT end user.

LAN Administration: This mostly deprecated term was once popular for referring to the small and medium business combined job role of deskside and server administrator along with network technician. LAN Administrators were often required to be “jack of all trades” functioning as a single point of resolution for all “computer” problems in small businesses. Often this meant constant trips to user’s desks and wrangling with tiny mixed user server and network closets. As IT advances this role is becoming less popular but is likely to continue in smaller companies for some time. The term LAN refers to the “Local Area Network” and was meant to suggest that the administrator was responsible for all machines connected to and including the office’s network. LAN Administrators also have a tendency to occur at remote, branch office locations where a single person can satisfy almost all local IT needs and additional needs can be handled via helpdesk or remote administration.

Storage Administration: One of the newest professional areas now widely available as a specialty within IT is that of storage management.  Over the last several years new and highly specific storage technologies have emerged and have become a mainstay in the corporate technology environment.  These technologies are, to some degree, unique to storage dealing with large and fast storage hardware as well as network technologies adapted for dedicated use in the storage space.  Storage results in generally being a blend of systems, networking and a little server level hardware support.  This is a young and growing area within IT but definitely here to stay.