PEF/REV: A Multi-dimensional bug tracking metric
Traditional one dimensional bug tracking suffers from a lack of directive capability. If each bug is rated "ReallyBad", "Bad", or "Trivial", no clear direction exists for which ones to fix first. If you concentrate on all the "ReallyBad" bugs, you may fix bugs that only 2% of your users encounter 1% of the time and it breaks other code. You may not get to "Bad" bugs that 90% of the users encounter 90% of the time.
The PEF/REV system addresses that problem. Each bug is rated with a PEF and a REV number.
- User's View: PEF - the sum of a rating for Pain, Effort, and Frequency.
- Pain
How painfully is this bug to a user. 1 - low pain, 3 - fairly painful, 9 - very painful
- Effort
How much effort does it take a user to work around the bug. A rating of 1 requires little effort, 3 means a fair amount of effort, and 9 implies a major amount of work.
- Frequency
How often does a user encountered this bug. 1 - infrequently, 3 - fairly often, 9 - frequently
The PEF represents the user's experience with the bug. A PEF of 27 means it's really painful, is very hard to work around, and happens all the time. A PEF of just 3 means it's not very painful, doesn't require much effort, and happens infrequently. All the really high PEF should be candidates to be fixed soon.
- Pain
- Programmer's View: REV - the sum of a rating for Risk, Effort, and Verifiability
- Risk
How risky is this fix to the system. 1 - little risk of collateral damage from the fix, 3 - some risk to other components, 9 - this fix touches lots of code and it's very risky to the entire system.
- Effort
How much effort does it take a programmer to fix this bug. A rating of 1 requires little effort, 3 means a fair amount of effort, and 9 implies a major amount of work.
- Verifiability
If we fix this bug, can we easily verify that it's fixed? 1 - it's easy to verify, 3 - we can verify with some amount of work, 9 - it's really hard to verify this bug is fixed.
Each bug also gets a REV number. A REV of 27 means it's a very risky fix which may endanger other code, it will take lots of programmer effort to fix it, and it's going to be difficult to verify the fix. High REV number bugs are bad.
The ideal bug (from a programmer's view point) is a PEF of 27 and a REV of 3. The worst case scenario is a PEF of 27 and a REV of 27.
- Risk
Using the PEF/REV system you can gain more control over releases by specifying a target goal. This goal might be that we are going to fix all bugs with a PEF over 21 and REV under 21. (This goal should not be immutable. Certain bugs will need to get fixed that are not in this category, and others we may not fix that fall into this criteria. The team should decide on deviations from the policy as a conscious decision.)