Connect Expo 2017

Connect Expo

The Connect Expo is on each year around late March in Melbourne at the Exhibition and Convention Centre. I had gone to previous events but this was our first time as an exhibitor.

 

Connect Expo - Successful Endeavours

Connect Expo – Successful Endeavours

This was by far the best Connect Expo I have been to. The mix of software vendors, web platform vendors, component suppliers and specialist IT vendors was excellent and there was also a specific section for eHealth. We set up our own IoT Platform demonstration with a QR code you could scan with a phone and take you to a webpage showing real time (less than five second delay) updates to the status of a device on the stand. A simple demo of the Internet of Things in action.

 

A really good trend I noticed was several Software Testing companies represented in the mix. Testing to confirm software is working correctly is a very important part of delivering a high quality product and it was good to see this coming through at the industry level.

 

We also shared the stand with Minnovation who do data science and analytics so it was also good to see how rapidly that area is expanding.

 

Successful Endeavours specialise in Electronics Design and Embedded Software Development, focusing on products that are intended to be Made In Australia. Ray Keefe has developed market leading electronics products in Australia for more than 30 years. This post is Copyright © 2017 Successful Endeavours Pty Ltd.

Software Documentation

Software Documentation

This is an area of Software Development that has always been a challenge. The documentation falls into six general categories:

 

  • Software Design Documentation
  • Software Testing and Test Results
  • Software Implementation Details
  • Software Change Management
  • User Documentation
  • Coding Standards

 

Each of these has its own specific issues.

 

There are formal documentation standards including IEEE 829 for Software Test Documentation and IEEE 830 for Software Requirements Specification and IEEE 1016 for Software Design Description. Even if you are not using these standards, it is worth reading them and understanding the ideas and methods they teach. These are well thought out standards.

 

Software Design Documentation

We work for a wide range of clients and some have very strict criteria while others leave it entirely up to us. The strictest criteria we work with are biomedical, automotive or transport clients. These require upfront definitions of everything including the Software Test Methodology and the specific tests to be done and the pass fail criteria. And of course, every change request up-issues every affected document. So that has to be budgeted for.

 

Software Design balances the project priorities

Software Design

For clients without specific requirements, we use the following methods:

 

  • Define the requirements and how they will be tested
  • State all known constraints
  • Describe the operating system
  • Show the Functional Decomposition into modules
  • Show the module communication
  • Use diagrams to show all State Machines
  • Document tests and test results

 

For larger projects, we might choose to have separate requirements and test documents but we always align the numbering so the test for a requirement has the same number as the requirement. For instance, if requirement 2.3.1 was that the system accommodate BAUD rates from 9600 to 115200, then test 2.3.1 would be to set and confirm the system operated with each of the BAUD rates.

 

This helps with both ensuring requirements are all tested and implemented, and that changes can easily identify which tests need review when a requirement is changed. Very large projects would use a Requirements Allocation Matrix where the requirement is cross-referenced with the modules that implement it.

 

Functional Decomposition

Functional Decomposition is the process of breaking the project up into specific modules and allocating requirements and functions to them. Since we do the whole gamut of Electronics Design and Embedded Software Development, this includes deciding how many processors we will use and how we break up requirements between them.

 

Functional Decomposition

Functional Decomposition

The intent is to take the complex and break it down into simpler pieces until they are simple enough to implement as functions in their own right.

 

Software Testing

For Software Testing the process is conceptually simple:

 

  • What is tested – which requirement are we meeting
  • How is it tested – the test protocol
  • What results are recorded – the test records
  • What the pass criteria is – the test acceptance criteria

 

This is usually best handled with tables except for user interface interactions which might use recordings as well as written test results documentation.

 

Software Implementation Details

This usually consists of the source code documentation since the Software Design has already been documented. Many modern toolsets include source browsing and database tools, but these require the end client to have the same toolsets. So we also add Source Code Documentation tags supported by Doxygen which allows a toolset and platform independent set of documentation to be created. If you haven’t used it, what you get is a website with everything hyper-linked. You can also create hyper-linked PDFs but we usually stay with the html.

 

Doxygen Source Code Documentation

Doxygen

You can create diagrams, caller graphs, callee graphs and state machine descriptions using the Graphviz tool which is supported by Doxygen.

 

Doxygen Call Graph

Doxygen Call Graph

Our gratitude goes to Dimitri Van Heesch for creating and maintaining Doxygen which you can support by donation.

 

Software Change Management

So you have a software project that passes all its tests and its out there and working. Now you, or your client, want to change something!

 

The last thing we want to do is break something while trying to make it better. So when changes are needed, they have to be analysed. Things to consider are:

 

  • Which requirements are affected by the change?
  • How risky is the change?
  • Who need to approve of the change?
  • Are there any new requirements or new tests required?
  • Will field testing be necessary?
  • Are there regulatory approvals affected?

 

So trivial changes like updating the wording on a form are usually low risk and don’t require a lot of risk management of multiple levels of signoff.

 

A more significant change like altering the wheel pressure balancing algorithm in a ABS braking system gets the maximum level of attention.

 

User Documentation

If the product has a User Interface or supports user interaction, then this also needs to be documented. This documentation is usually written for the user to read and is intended to assist the user.

 

The format of the documentation depends on the product. It could be online help, tool tips, printed manual, soft manual, on screen or printed on the product itself. One method of User Documentation assessment is with randomly selected people who could be potential users. This is to identify concepts understood by the development team by not either generally understood or else not adequately explained in the documentation.

 

Coding Standards

This will be covered in more detail in another blog post. Because a range of programmers will work on any significant project, it is well worth defining how code is to be written. This consistency will lead to easier to read and maintain code.

 

For instance, Embedded C bracketing uses to follow the UNIX convention which goes like this:

 

for(length = MIN; length <= MAX; length++){
do_some_stuff();}
next_lot_of_stuff_to_do();

 

This had advantages in the days when VDU screen space and Teletype or Lineprinter paper space was at a premium, but it makes sense to make the indenting easier to understand by doing it this way:

 

for(length = MIN; length <= MAX; length++)
{
do_some_stuff();
}
next_lot_of_stuff_to_do();

 

Regardless of the type of project, easy to read and understand Software Documentation will reduce maintenance costs is essential for larger teams to be able to deliver a working project.

 

Successful Endeavours specialise in Electronics Design and Embedded Software Development. Ray Keefe has developed market leading electronics products in Australia for nearly 30 years. This post is Copyright © 2012 Successful Endeavours Pty Ltd

Squash Lessons for Engineering

Squash Lessons for Engineering

The picture in today’s post comes courtesy of Dr Marc Dussault, The Exponential Growth Strategist. At his recent Exponential Business Building Bootcamp, he demonstrated how a Squash Racquet gets broken from repeated use.

 

Broken Squash Racquet

 

So what does this have to do with Engineering? Glad you asked.

 

First, I have to explain the demonstration. Marc showed that it takes a very large amount of force to break the Squash Racquet. He really applied himself to the destructive task and it took a few minutes of escalating Squash Racquet abuse before it finally succumbed and broke. Some of us in the front of the room could tell just how much it required to break the Squash Racquet. However the Squash Racket already had a crack, so Marc knew where to apply the force in order to break it. The picture above is the final outcome. Without the crack being obvious, it would have been almost impossible to have broken the Squash Racquet using just randomly applied force.

 

Marc then explained that way the Squash Racquet became cracked in the first place, was by it being consistently scraped along the wall as he retrieved the ball from shots along the wall. Marc is an outstanding competitive squash player and currently ranks as World # 18! So he knows his stuff when it comes to squash. You can read more about this at his Mindset Of A Champion blog.

 

So if you know what to look for, you can monitor the thinning of the racquet and get an idea of when and where it might fail. If you don’t know what to look for, then the failure will be unexpected.

 

Software Testing and Software Engineering

A lot of Software Testing can suffer from the same problem. If you already know where the weakness will be and how to spot it, then finding a bug is easy. You can set up the scenario, monitor for the symptom and confirm the failure. Or, if you have enough resources you can go the brute force approach and just break it through the persistent use of randomly directed and escalated force of testing. However very products are simple enough and very few companies are large enough to have that level of resource and to solve the problem this way. So for the rest of us, the other 99.995%, a more intelligent approach is needed.

 

Since you don’t know where and when it will fail, it is best to remove failure causes from the beginning. This is where Software Engineering come is. Software Engineering is not just coding. Coding is the production end of the Software Engineering process. Software Engineering is about designing the system so you have defined the components so they are each fully testable in their own right. Then you can apply processes like Unit Testing to ensure they are fully functional as stand alone pieces of software. You can then perform Integration Testing to ensure that software added to the system correctly handles both the Execution Flow, also known as Control Flow, and Data Flow required including error and Exception Handling. The result is that you build up a fully working and correctly executing system quickly and with great confidence. It isn’t a magic bullet but it is close to it.

 

As was famously quipped by Edsger Dijkstra, “If Debugging is the process of removing bugs, then programming must be the process of putting them in”.

 

So if you put less bugs in, you have less debugging to do. And that saves time and removes future time bombs. Because the chance that you find them all is zero percent. And you can’t create a system that is 100% testable by brute force means. So you have to go about it smarter. It will save time, money and improve the business outcome now and into the future.

 

Ray Keefe has been developing high quality and market leading electronics products in Australia for nearly 30 years. For more information go to his LinkedIn profile. This post is Copyright © 2010 Successful Endeavours Pty Ltd.

Embedded Software Testing

Software Testing

I recently met with an Australian Software Development company, PepperStack, and we got onto the subject of Software Testing. As someone who began their career as an Electronics Hardware Engineer, one of the things I learnt was that you have to test thoroughly to be sure everything is working as it should be. With Electronics, if you make a mistake with an Engineering Calculation you can easily destroy things. This is sometimes referred to as “letting the smoke out”. So it was good to meet with others who believe in the same level of rigorous software unit, module and system testing that we do.

 

 

Some Engineering Humour

Which reminds me of a joke I once heard:

 

There are three Engineers in a car going for a drive. The first is a Mechanical Engineer, the second an Electronics Engineer and the third is a Software Engineer. Fortunately the Mechanical Engineer is driving because the brakes fail and they are going downhill. The Mechanical Engineer eventually brings the car safely to a halt and gets out to examine the hydraulic systems. The Electronics Engineer gets out and checks and body computer, ABS system and the power train CAN bus. The Software Engineer stays in the car and when queried about it says that they should all just get back in the car and see if it happens again!

 

Now don’t get me wrong, I’m not having a go at Software Engineers. The process of finding and eliminating faults is a very important part of the development cycle and is something that needs up front thinking and not just responding to symptoms. And the more complex or sophisticated a system is, that harder it is to fully test every possible response to every possible stimuli and after a certain point it becomes impractical to have 100% Test Coverage (every line of code has been executed through all of the possible states). The reason this is a bigger problem with Software Development is that the flexibility of software means that it is inherently complex and it takes skill and planning to manage that complexity so it is testable.

 

So here is the issue. More than any other discipline, faults can be experienced by an end user of a product under a situation or scenario you could not have proactively tested against before release. There are many potential reasons for this including:

  • change of hardware or operating system environment
  • new standards or protocols
  • the sheer number of potential combinations of drivers, peripherals, software and users
  • the product being used for a purpose it wasn’t originally designed for
  • gamma ray corruption of a memory location – I am getting esoteric now but in some areas like avionics and space this is a big threat

So how do you reduce the likelihood of these problems occurring?

 

 

Improving Software Quality

With many new products having Electronics and Embedded Software and the Software Development requiring 80% of the effort, it is important to delivery it as quickly and fault free as you can. The main weapons in your Software Quality arsenal have been known about for a long time but are, in our experience, just not used. These are:

  • Architectural Design – work out how the data and execution flow will happen and how you will manage the constraints
  • Functional Decomposition – divide and conquer but with an emphasis on how each module fits into the system and how the interfaces work in detail
  • Error handling – who will decide what to do with response codes – again this is data and execution flow and part of the architecture. In many cases exception management is at least 50% of the project.
  • Have an Integration Test Plan – some thing that proves the data and execution flow matches the architectural design. Too often “it builds” seems to be good enough here.
  • Unit Test modules – so you remove all the issues before adding them to the integration
  • Do the Integration Tests before you try system testing
  • Design modules so you can integrate them as shells then add functionality down the track
  • Have NVM and configuration data available at the beginning of the project and not as an after thought at the end
  • Have a System Test Plan and use it
  • Use some of the good practices of Test Driven Development – run the tests every time you change the code
  • Have a rationale for what level of Code Coverage you can accept
  • Have a rationale for what level of Churn you can accept – Churn is the percentage of the lines of code that have changed in the past time period. Usually either a week or month depending on the size of the project.
  • Use automated software quality tools. For instance we use both PC-Lint and RSM to automated many software quality metrics which saves a lot of time in Code Reviews
  • Use Code Reviews, also known as Software Peer Review. It really does save time.

Next I plan to look at what you can learn about software testing from a Squash Racquet.

 

Ray Keefe has been developing high quality and market leading electronics products in Australia for nearly 30 years. For more information go to his LinkedIn profile. This post is Copyright © 2010 Successful Endeavours Pty Ltd.