Author: Joe H., Inflow Engineer
As I discussed in last week’s post, Choosing the Right Metrics, developing good metrics requires a detailed understanding of the problem you are trying to solve. However, there are plenty of reasons why precise metrics can’t be generated. Maybe the system you’re designing will be used in an environment you can’t predict, or maybe you just don’t know enough about the problem you’re trying to solve. In this post, we’re going to take a look at how an engineer approaches a problem where good metrics can’t be defined.
Like all the other engineering techniques we’ve covered, our approach to dealing with ill-defined or non-existent metrics is going to focus on reducing the complexity of the problem. We know how to use iteration and black boxing to design systems with fully-defined metrics and requirements, so instead of figuring out a whole new approach, we need a method that lets us use the tools we already have. To do so, we need a way to convert metrics like “I need this car to be fast” into something more usable. The engineering solution is to use something called a factor of safety.
A factor of safety is essentially a requirements multiplier. When you have a very well defined metric, you don’t need much of a factor of safety, since you know exactly how your design will be used. For example, if you’re designing a light fixture, you know the fixture will be hooked up to a power source with a specific voltage and current, protected by a fuse box, so your power wires don’t need to be designed to deal with anything beyond that voltage. You can run the numbers, figure out how much insulation and what wire gauge you’ll need, and move on with your design. If you’re designing the fuse box, you know that you’ll normally be getting a certain voltage and current, and that sometimes you might get a surge, but there’s no good way to know how big that surge might be. You could do some research into typical power surge characteristics, but you might still get a larger surge. However, if you take the characteristics of a typical power surge and multiply them all by a factor of four or five, you’ll have a set of requirements which should cover a vast majority of possible power surges. That multiplier is the factor of safety.
The goal of using a factor of safety isn’t to generate a perfect set of requirements, but rather to create metrics that are good enough to be used. The less confident you are about the type of demands that will be placed on your design, the larger you make your factor of safety. When you have a more solid handle on the requirements, a large factor of safety will result in costly over-designing, so you use a much smaller factor, but even a design based on very well defined requirements will typically have a small factor of safety just in case something unexpected happens to the system. In either case, you’ve taken the uncertainty out of the design requirement, converting a complex problem into a simple one.
Within engineering fields, there are best practices regarding exactly what factor of safety you should use for a given design problem, but when you are applying the concept to a general problem, you’ll probably have to make a judgement call. One common real world scenario is providing an estimated completion date on a project. Typically, no one gets upset if you finish a project early or under budget, so it’s better to err on the side of caution. If it’s something that you’ve done before and are comfortable with, you should have a good idea of how long it will take, so whatever you estimate probably doesn’t need to be multiplied by very much. However, for a new project even your best guess might be off by a good deal. Again, the less confident you are in your estimate, the larger your factor of safety should be to compensate.
The concept of using what is essentially a fudge factor is certainly not unique to engineering, but engineering is, as far as I know, unique in formalizing the approach. In order to apply the concept like an engineer would, you need to be continuously analyzing how accurate your factor of safety estimates are. In our last example, if you typically multiply your time estimates by three for new projects, but consistently finish a week earlier then you estimated, you need to lower your factor of safety. With a little bit of effort, you’ll develop a powerful tool for accurately accounting for uncertainties in your work!
-Joe H., Inflow Engineer
At Inflow we solve complex terror and criminal issues for the United States Government and their partners, by providing high quality and innovative solutions at the right price through the cultivation of a corporate culture dedicated to being #1 in employee and customer engagement. We Make it Matter, by putting people first! If you are interested in working for Inflow or partnering with us on future projects, contact us here.