Software Development Metrics

Software Development Estimation

in Software Costing we looked at some typical ways of determining the likely cost of a Software Development project. One way was to look at how much typing was involved. This is not considered the best approach as the cartoon below demonstrated.

 

Software Estimation

Software Estimation

I prefer to spend more time thinking about the problem and less time typing. Among other things this reduces the amount of typing required.

 

The other example was by looking at the study by VDC Research where the median Software Development project had a cost of US$12.50 per line of code. Which is also very expensive by our reckoning. But these are actual figures. The hard part is working out how many lines of code (LOC) will be required before you have written them.

 

Estimation by Function Points

So I was interested when The Embedded Muse, a software development newsletter authored by Jack Ganssle, published some data from Capers Jones who is a legend in the Software Development industry for his statistics on the Software Development Process.

Capers Jones & Quote

Capers Jones & Quote

My first thought was that I fully agree with the quote. Let’s have a look at it in detail.

High-quality software is not expensive. High-quality software is faster and cheaper to build and maintain than low-quality software, from initial development all the way through total cost of ownership.

 

The assertion is that if you have a way to measure Software Quality, and you stick to good Software Development Process and the creation of High Quality Software, the overall cost will be lower. This might seems counter intuitive to some of our prospects given the development process requests we get, but I think it is spot on. Here are the other items of interest.

 

According to Capers Jones, a very rough guide to estimating the number of people needed on a project, and the project’s duration, is:

 

  • Number of developers = (function points)/150
  • Calendar months = (function points) x 0.4
  • One function point is approximately 130 lines of C Code

So for a typical 20,000 lines of C Code project, we would expect 154 function points with 1 developer required and it would take 5 years. If I look at one of the Industrial Control projects we did then that was 60,000 lines with 3 developers (that part works) and 8 months overall (rather than 5 years). So I’m not convinced the time component of the formula works. Maybe it is too heavily influenced by larger projects. But the number of developers seems to be about right.

 

Again, these are actual project figures and not some abstract model. And if you don’t use good Software Development Process then the cost of debugging and fixing issues in the field will typically dwarf the cost of writing the code.

 

Software Development Metrics

So I thought I’d do a summary of the Software Development Metrics we have accumulated over time as this may help with the Software Estimation process. This is what it looks like in our world of typically smaller projects (less than 100K LOC) reflected through the data above:

 

  • Number of developers = LOC  / 20,000
  • Number of months = LOC / ((number of developers) x 6600)
  • Cost of development = $4.80 per LOC

This assumes good process and average project difficulty. Things that will push a project way past this are code space constraints, processing power constraints, poorly defined requirements and substantial changes of direction mid project. But it does give some way to at least know the ballpark.

 

Successful Endeavours specialise in Electronics Design and Embedded Software Development, focusing on products that are intended to be Made In AustraliaRay Keefe has developed market leading electronics products in Australia for more than 30 years. This post is Copyright © 2018 Successful Endeavours Pty Ltd.

Embedded Software Testing

Software Testing

I recently met with an Australian Software Development company, PepperStack, and we got onto the subject of Software Testing. As someone who began their career as an Electronics Hardware Engineer, one of the things I learnt was that you have to test thoroughly to be sure everything is working as it should be. With Electronics, if you make a mistake with an Engineering Calculation you can easily destroy things. This is sometimes referred to as “letting the smoke out”. So it was good to meet with others who believe in the same level of rigorous software unit, module and system testing that we do.

 

 

Some Engineering Humour

Which reminds me of a joke I once heard:

 

There are three Engineers in a car going for a drive. The first is a Mechanical Engineer, the second an Electronics Engineer and the third is a Software Engineer. Fortunately the Mechanical Engineer is driving because the brakes fail and they are going downhill. The Mechanical Engineer eventually brings the car safely to a halt and gets out to examine the hydraulic systems. The Electronics Engineer gets out and checks and body computer, ABS system and the power train CAN bus. The Software Engineer stays in the car and when queried about it says that they should all just get back in the car and see if it happens again!

 

Now don’t get me wrong, I’m not having a go at Software Engineers. The process of finding and eliminating faults is a very important part of the development cycle and is something that needs up front thinking and not just responding to symptoms. And the more complex or sophisticated a system is, that harder it is to fully test every possible response to every possible stimuli and after a certain point it becomes impractical to have 100% Test Coverage (every line of code has been executed through all of the possible states). The reason this is a bigger problem with Software Development is that the flexibility of software means that it is inherently complex and it takes skill and planning to manage that complexity so it is testable.

 

So here is the issue. More than any other discipline, faults can be experienced by an end user of a product under a situation or scenario you could not have proactively tested against before release. There are many potential reasons for this including:

  • change of hardware or operating system environment
  • new standards or protocols
  • the sheer number of potential combinations of drivers, peripherals, software and users
  • the product being used for a purpose it wasn’t originally designed for
  • gamma ray corruption of a memory location – I am getting esoteric now but in some areas like avionics and space this is a big threat

So how do you reduce the likelihood of these problems occurring?

 

 

Improving Software Quality

With many new products having Electronics and Embedded Software and the Software Development requiring 80% of the effort, it is important to delivery it as quickly and fault free as you can. The main weapons in your Software Quality arsenal have been known about for a long time but are, in our experience, just not used. These are:

  • Architectural Design – work out how the data and execution flow will happen and how you will manage the constraints
  • Functional Decomposition – divide and conquer but with an emphasis on how each module fits into the system and how the interfaces work in detail
  • Error handling – who will decide what to do with response codes – again this is data and execution flow and part of the architecture. In many cases exception management is at least 50% of the project.
  • Have an Integration Test Plan – some thing that proves the data and execution flow matches the architectural design. Too often “it builds” seems to be good enough here.
  • Unit Test modules – so you remove all the issues before adding them to the integration
  • Do the Integration Tests before you try system testing
  • Design modules so you can integrate them as shells then add functionality down the track
  • Have NVM and configuration data available at the beginning of the project and not as an after thought at the end
  • Have a System Test Plan and use it
  • Use some of the good practices of Test Driven Development – run the tests every time you change the code
  • Have a rationale for what level of Code Coverage you can accept
  • Have a rationale for what level of Churn you can accept – Churn is the percentage of the lines of code that have changed in the past time period. Usually either a week or month depending on the size of the project.
  • Use automated software quality tools. For instance we use both PC-Lint and RSM to automated many software quality metrics which saves a lot of time in Code Reviews
  • Use Code Reviews, also known as Software Peer Review. It really does save time.

Next I plan to look at what you can learn about software testing from a Squash Racquet.

 

Ray Keefe has been developing high quality and market leading electronics products in Australia for nearly 30 years. For more information go to his LinkedIn profile. This post is Copyright © 2010 Successful Endeavours Pty Ltd.