Musings of an IT Manager…
Inefficiency in software development is an oft heard term and a beaten track through the various models of software engineering process that is attempting to herd the beast into a possible single lowest denominator of predictable efficiency.
It is equally well stated that highly competent individuals, or teams of high caliber if they hit the right note, make it ridiculously simple to turn around a complex project in the least time and overall costs. The trouble is in scaling it and performing consistently.
In most instances, the issue is of herding a team of probably incongruous members, inadequately skilled staffing and in all probabilities less than optimal plan, due diligence of requirements in place. Highly structured processes and metrics are good crutches for the average teams to depend on to improve the predictability and manage the risks proactively rather than react to circumstances late. Processes and metrics that are optimally structured yield essential benefits, even for highly competent teams.
For the efficient and competent, I believe the ability to adapt and proactiveness being high over and above the skills being beyond doubt, the process and metrics are mere annoying statistics, but can also give the morale or the ego a rough nudge.
Processes and metrics are a must from a management perspective, but when and how the lines are drawn to ensure they do not become an albatross around the team’s neck, that effectively sheds competent members and leaves it staffed with less than the most capable.
It is not a secret to be ferreted out that the capable and imminently most knowledgeable engineer will shirk from being too flexible, bend to a management diktat, where the less capable will willingly bend and twist to the needs of the hour.
Concept of a factory production applied to engineering work floors, hinges on the fact that the repeatability of the process can affect economies in the kind of person employed, materials used and overall time to produce being minimized. Over competence is a negative quality in those circumstances, which will be tolerated only in specific situations. Right fitment being the key word, remains an euphemism to staff less than capable teams and put in even more management effort in training and adapting them to the tasks to be performed.
There we should realize are the two opposite poles for applying process and metrics controls in the context of software projects.
Software engineering is not “engineering” and Computer Science is not a “science”. For the word engineering indicates the possibilities of scale economies, ordered processes, tasks repeatable over long time intervals in the order of years which is not the case. Science as a term would render it closer to the pure sciences of maths, physics, chemistry etc.. that it barely can profess to be except in the narrow circles of Information Technology academia, where too if they are not succumbing to the market forces of company fundable research in current technologies rather than more pure science oriented research. Information Technology seems a lot more appropriate as it brings together the concept of technology domain in the manipulation, service of information or data. Keeping this in context it would be appropriate to evolve methods and means to extract the optimal productivity that fit this context, not that of engineering or science. Similes of urban planning for enterprise architecture process appear a close fit but misleading too as the tangible outputs of a civil architectural process is a much more easily quantifiable and discernable output for any measurement, controls than an intangible mass of code lines that till they are complete and functional yield to subjective personal judgments at best. Milestone and modular segmentations yield better comprehension but have historically proven to be misleading till the final integrated product is tabled.
Most if not all concepts of Agile, CMM process look downright appealing in theory and in practice stumble over the human elements not accounted for. Through the history training has been the silver bullet prescribed to alleviate the human capability or skill diversity, yet it remains to be an effective closure to the project deliverability. Primary if not the major reason being the human inability to reconcile to throw away the yokes of their accepted traditional ways and means of working on their tasks, change to hook up together as a team
Predictability, Complete Functionality, Productivity, Price and Performance Efficiency being the overriding goals in any business enterprise context absolutely applicable for software projects also.
Requirements, Design, Development, Testing, Release, Documentation, Delivery the key components of the software project lifecycle.
Requirements gathering and elucidation to ensure as close a fit to final shape of the product desired is mandatory, but rarely occurs close to an engineering process. The management or the customer perception that code lines being an easily modifiable entity unlike a civil superstructure or an automobile workshop production line is a bane to the project process. The requirements variance should be tracked and reflected in the early stages itself to ensure that productivity after effects are clearly visible before the changed requirements percolate to the design level. Change management should kick in early in this lifecycle to track this variation and its impacts down the line both tangible and intangible. Traceability of economic impacts of requirements variance should be more honed and definitive to assist better judgment.
What this would beget is an analogous process of rapid prototyping, complete design and development specifications before an implementation is initiated at a project level. In effect reduce the gap in requirement variation over shorter intervals of time. The critical difference from an engineering perspective is that the product is not being mass produced 100,000 times over the next few years. It is produced once and duplicated on a media probably 100 to 100,000 times, so the implementation of the design/development spec happens just once. Changes in the technology happens rather more rapidly now that impacts the state of decision making even as short as half year down the project timeline, when something else starts to appear more appealing an alternative for productivity and feature set reasons, derailing if adopted the predictability of the project schedule and costs and moreover affecting intangibly most times the morale of the team in a subtle manner. How far the projects can be broken up into smaller manageable release cycles is also moot and entirely dependent on the business domain, but would certainly help if a core is released and features added in subsequent cycles thereof.
Unless there is standardization of technology, mid-term deliverable quantification and assessments, firms that adopt high level of process standardization and automation for highly repeatable nature of projects that are handled it would be near impossible to fit one method, process, and metrics to suit all kinds, sizes, nature of projects to be executed. The challenge then is to have enough on the plate to justify economics of operations year on year..
The alternative thereof is the option of agile process that defines everything as changeable and human ability to adapt as paramount. Nothing is fundamentally mandatory and defined.
Can there be a twist more to this agility and say that the initial and start up run of the project duration should be done in good faith without cost being charged to the customer and only if he is convinced of the utility and productiveness of going forward with the vendor is the charge to come into effect. Eliminate in so far as possible the incapability, lack of skills issues and proven capability being the hard rock for further engagement on a per project basis. Somewhat akin to our taking a test drive before buying the car. Even a fighter aircraft needs to be proven by hours of demonstration flights before it can be a prospective state purchase, why not for software projects that show a minimal period of time of productive engagement before charging the customer for the project.
Time and Material basis for executing projects, in the nature of theoretical basis, appears as the most genuine, least mark up mode of executing a project. But in reality is nothing short of bartering human capital in exactly the same manner as cattle is traded. The expertise, skills, experience are all white washed in a measure that renders staffing a game of numbers, given the volume of skilled resources required globally.
Fixed price bids will work only if there is high level of due diligence from the customers end as much as is done from the vendor’s end. Given the risk involved and ease at which even now customers accept engineers staffing rather than sub-project contracting, services organizations will prefer only the T&M mode till they are forced to switch.
If an only if the customers demand an effective competitive bid for projects and the ensuing competition forces services organizations to quote precisely, the software projects shall continue to languish in its state of chaos, with occasional and the minor successes hailed as personal victories. Fixed price bids will then raise the stake for utilization of better more competent engineers and in more appropriately structured manner even for them to contribute within the framework of governed processes. The deliverables will be more accurately measured and the milestones better tracked for the sheer need to do so to contain the risk of losses.