Metrics, nowadays has gained importance in any matured IT organization to evaluate process effectiveness, understand situation, track progress and more. Efficient test process measurement is essential factor for success of any project, we generally measure many aspects of test process but the gap between how & what DO we measure and how & what COULD we measure has remained larger than what it should be. The key reason for this gap is lack of coordinated and comprehensive framework for understanding and using measurements. Many times, metrics and measurement are done just for sake of doing and less importance is given.
“You can’t control what you can’t measure.” A quote form To DeMorco’s book, controlling software projects is very true. Testing is an activity that needs effective controlling and measurements in order to understand where we are heading. Measuring is nothing but recording of past things to quantitatively predict future things. Metric is a measurable indication of some quantitative aspects of system/process. Any Testing project or activity involves many things like, test plan, test design, test development and execution. Test metrics helps to measure these aspects of test activities and provide heads-up to raise voices for faster, more informed decision making.
I. Lack of strong Governance: There should be strong governance in place in term of owning and driving the initiatives on ongoing basis. Management should be committed in what they initiate and should continue practicing throughout the program. Discontinuing in later stage will affects and demotivates the team and hence become reason of failure of metric program. Due Importance should be given for metrics program.
II. Unclear Metrics definition: Unclear metric definitions can be dangerous, as different people may understand them in different ways, thus resulting in inaccurate outcomes. Metric should be defined clearly with clear goals associated with it. It should pass on the clear message of what needs to be collected and why need to be generated.
Ignorance in communicating expectations: Ignorance in communicating expectations and importance of data needed for better control and effectiveness. Explain why do you want to measure the item you have chosen, how will it be useful, everyone has right to know why is something requested. All the data which is collected should be used effectively.
IV. Practice of not sharing the results: There should be good practice of sharing the results or metrics generated after collecting the data. More often team thinks that data collection is boring activity and overhead on their current process. Team members will be more motivated to participate in the measurement activities if you inform them about how you have used the data. Share summaries and trends with the team at regular intervals and get them to help you understand the data. Let them know whenever you use their data to answer a question, make a prediction or a decision.
V. Attitudes of people: Attitudes of the people involved in collecting data, sharing, calculating, reporting and using metrics plays important role in success or failure of program. Accurate and meaningful Metric generations are very much dependent on Individuals approach and attitudes. For some people it’s just a waste activity and meaningless. They often think this is just way to show good stuffs to clients/Sponsorers and make an impression. People want to look good and hence they show only good measurements rather than actual. The best way to avoid human factors in measurement are to,
- Avoid measuring individuals:It is often taken at personal levels when metrics are collected and measured against individuals.
- Use of Metric to motivate teams:One should not use metric or pass on a message where team thinks that they are being motivated to do better, this is wrong message and team starts showing unsupportiveness and provide false data.
- Providing feedback:Providing continuous feedback on the data which is collected has several benefits like, when team sees the data is actually being used, they are more likely to consider and give importance to data collections activities. By involving team members in data analysis and process improvement efforts, we benefit from their unique knowledge and experience; the benefit can be more accurate, consistent, and timely data.
VI. Identification of several metrics: Identification of several metrics than Identify those metrics which are necessary and adds value in current process. Some test lead/managers urges to collect the data and generate many metrics which are not in line with the goal and intention. There should be practice of collecting only those metrics which are relevant and in line with goals, definitions.
VII. Lack of communication and training: VII. Proper communication on what and why we need metrics is the key to the success of the metrics program. All members need to understand the relevance of any data that is collected and this can only be achieved by educating team on what is being collected, how it will be collected and how it will be used. There should be continuous feedbacks on the data being collected. Proper trainings and communications plays important role in successfully implementing metrics model.