OpenSource Connections
Category: Uncategorized
We are not a CMMI rated company. As a small business, we think it is more important to focus on delivering excellence for our customers and growing top and bottom line numbers than being able to check certain boxes about whether or not we have a laundry list of processes documented and implemented. The causation seems to be backwards. We do things which please our customers. Our customers are pleased; therefore, we seem to be doing the right thing, QED. Having a CMMI certification does not guarantee anyone that we will perform in a manner which will satisfy our customers, only that we will perform in a manner which will ensure we can check boxes.
What is CMMI, you ask? According to the Carnegie Mellon Software Engineering Institute CMMI page:
“Capability Maturity Model® Integration (CMMI) is a process improvement approach that provides organizations with the essential elements of effective processes. It can be used to guide process improvement across a project, a division, or an entire organization. CMMI helps integrate traditionally separate organizational functions, set process improvement goals and priorities, provide guidance for quality processes, and provide a point of reference for appraising current processes.”
A sample of the 23 different areas that CMMI measures includes (quoted from the CMMI v 1.2 manual):
· Causal Analysis and Resolution (CAR)
· Configuration Management (CM)
· Decision Analysis and Resolution (DAR)
· Integrated Project Management +IPPD (IPM+IPPD) [1]
· Measurement and Analysis (MA)
· Organizational Innovation and Deployment (OID)
· Organizational Process Definition +IPPD (OPD+IPPD)
· Organizational Process Focus (OPF)
· Organizational Process Performance (OPP)
· Organizational Training (OT)
For an organization which is not delivering the intended results to its customers, revising and revamping the processes is a good thing. I by no means wish to say that I do not believe in process. Process, within reasonable limits, is good. We are proponents of Agile Development processes. With a complete lack of process, a project will end up with variable timelines, bugs, few (if any) test, and, most likely, a product that suits the developers needs rather than the customers needs.
To me, the question is how many companies who are CMMI-X level certified companies actually implement the CMMI processes fully rather than just showing the process in a project in order to get the certification? As Dr. Rick Hefner states (p.7), “A CMMI appraisal indicates the organization€™s capacity to perform the next project, but cannot guarantee that each new project will perform in that way.”
According to this Washington Technology article, the trend in government contracting is to mandate CMMI maturity levels. So, government contractors who wish to do more business with the government look to get CMMI certified so that they can win more contracts. The government thinks that enforcing certain process standards will yield more predictability in results.
Yet, this does not always happen. Government contracting officers and IT managers with whom I have spoken complain that CMMI level 4 and 5 companies are consistently delivering poor code bases, usually over budget (and/or late), and often full of critical bugs that are introduced into live environments.
Where is the disconnect, then? It happens from both sides.
Providers:
- Focus on the customer: The prettiest code or the coolest algorithm is worthless if it does not deliver what the customer wants. The scrum master and the development team should be in harmony with the product owner and business customer (if they are not the same person) in terms of vision and outcomes.
- Test, test, test: Code should not be released into a live environment unless it is unit tested, string tested, integration tested, and user tested. Fixing the nth incremental bug at a negative marginal utility cost is not the goal. Catching the critical bugs early is the goal. If the team focuses on test-driven development, then they will eliminate much of the overhead necessary to deliver working products at the end of the project.
- Prototype early: By developing form and function prototypes, the development team can give the customer an idea of what is going to be delivered, and the customer can provide intervening feedback to keep the development team pointed in the right direction. Prototyping reduces the amount of intent interpretation necessary, as the customer gets to see what the results will look like rather than hoping that the development team can divine intent.
- Dont drown in unnecessary documentation: Each report that the team has to deliver reduces the amount of time that the team can spend coding. Stick to burndown charts and product backlogs as much as possible. Scrum masters should push back where possible on excess reporting. Providing periodic performance updates is appropriate; crafting 800 page requirements documents is rarely so. Dont shirk from documenting code, though. Someone has to figure out what the team did long after the team is gone.
Customers:
- CMMI certification is not a magic bullet: Dont expect that just because a company has a CMMI certification, theyre going to magically deliver under budget, early, and with more scope than you thought. Expect the appropriate amount of coding rigor to deliver on what you need. Ask where the code tests are. Demand prototypes. Own the product backlog.
- Keep a gentle, guiding hand on the team, not an oppressing fist: Just as not enough oversight ruins the outcome, so does micromanagement. Expecting TPS reports every week means that youre going to have a team focused on filling out satisfactory paperwork rather than delivering outstanding code. Give guidance, vision, and direction; expect rigor, and get out of the way.
- Provide performance-based incentives: Periodic payments should keep the lights on with your provider. All of the profit should come at the end when the product delivers what it is supposed to, on time, and meeting the customers intent. Hourly fees are an invitation to print money. Even if the long-term total cost is unknown, set up per sprint requirements and hourly estimates and then hold the provider to those. For example, for a 3 year project, set up monthly (or whatever appropriate interval) sprints and get agreement on sprint deliveries and cost for each sprint, while holding out a percentage of the overall compensation to the end of the project contingent upon mutually agreed upon final deliverables. If those need to change, then negotiate the deliverables through the course of the project.
- Look at other customers besides those provided by the contractor in the proposal: Requests for proposal are inherently subject to selection bias. As a contractor and proposal provider, we are going to pick our 3 happiest customers and provide them to the requestor. Look deeper into the picture and see if all of the stories are the same. Request a dissatisfied customer in the proposal and dig into why the customer was dissatisfied. Nobody (not even us) has no unhappy customers. Everyone makes mistakes. Find out if the mistakes are systemic or have been corrected before jumping into the long-term relationship.
- Try before you buy: Do small contracts at first. Build the relationship and the rapport with the potential provider. See if the product is as good as the promise that came in the proposal.
- Dont be afraid to switch providers: Go with your hunches. If things seem to be going off course, then they probably are. Look at what youve spent as a sunk cost and move on and find the right solution. Staying in a bad marriage is not the answer in life, and its not the answer in business.
In Malcom Gladwells book Blink, he relates the story of a Pentagon exercise conducted in 2002 called Millenium Challenge. The Blue team, the Allies, were to go against the Red team, a Middle Eastern dictatorship. The Blue team, with its superior information, and its abundance of processes, was supposed to destroy the Red team. What they failed to account for is how their processes got in the way of actually doing anything, and while they were busy writing reports and doing prediction scenarios, LTG Paul van Ripers Red team was actually doing things. They were agile. The Blue team was process-obsessed and process-drowned, and they lost. Streamlined processes work. Process for the sake of process makes a team focus only on the process and not on the outcome.
So, having a CMMI certification may be good, but only if done in synch with an organizational focus on serving the customer in the most efficient manner possible. Having a CMMI certification only to check a box or to make sure that boxes are being checked during a project so that the project lead can cover his or her tracks is not. Id rather focus on the customer and getting things right and delivering a solid product. If it means that somewhere along the way, we find that we could be CMMI certified because of the way we do business, then so be it. After all, as Jim Nist pointed out to me, software development is hard.