Velocity
•
April 1, 2024
.png)
Engineering Managers have a variety of metrics they can draw from to assist in understanding their teams' efficiency & operational performance. What’s missing is a precise measurement of Risk (i.e. complexity) and Resilience (i.e. mitigation effort).
These dimensions rightfully categorize changes so that macro level metrics like DORA can be acted upon with precision.
Shepherdly Metrics Explained
Shepherdly introduces a novel approach to measuring the risk associated with pull requests and the mitigations necessary to ensure resilience. At its core, a predictive model is used to assess the risk of introducing bugs through code changes.
The model calculates a Risk Score, which then determines the Resilience Coverage needed—essentially, the mitigations required to guard against potential bugs. This approach not only quantifies the risk but also guides developers on specific actions to enhance codebase resilience.
Comparative Analysis
Risk Management and Mitigation
While Shepherdly zeroes in on the risk at the pull request level, providing actionable insights for mitigation, DORA offers a more macro perspective. DORA's Change Failure Rate, for example, assesses the overall risk posture of the deployment process but does not offer granular insights into individual code changes or categorizes change failure rate by risk.
Resilience and Reliability
Shepherdly's Resilience Coverage actively promotes proactive mitigation where it’s needed, encouraging developers to implement best practices directly in the PR. In contrast, DORA's Time to Restore Service focuses on the team's ability to respond to and recover from incidents, highlighting operational resilience rather than change resilience.
Efficiency and Productivity
Since most changes are objectively lower risk, there is justification to treat these differently in the review process by accelerating their deployment. While DORA offers insight into Deployment Frequency and Lead Time for Changes, Shepherdly disambiguates the driving factors by its risk classification.
Furthermore, by measuring the resilience of a change, teams can isolate which areas of their development process are contributing to Change Defect Rate.
Strategic Decision-Making and Visibility
By capturing innate risk and resilience (a proxy for effort), teams can have a nuanced understanding of how to balance speed & resilience before changes are shipped. For teams seeking deeper automation, the desired balance can be codified per repository.
DORA metrics guide broader strategic decisions around DevOps practices and investments, providing visibility into the health and performance of the software delivery lifecycle.
Main Takeaway
Having a broad understanding is great but typically leads to questions that require deeper analysis in order to act. Shepherdly is the tool that allows you to identify the main drivers behind DORA metric trends.
| Category | Examples | Collected |
|---|---|---|
| A. Identifiers | Contact details, such as real name, alias, postal address, telephone or mobile contact number, unique personal identifier, online identifier, Internet Protocol address, email address, and account name | YES |
| B. Personal information categories listed in the California Customer Records statute | Name, contact information, education, employment, employment history, and financial information | NO |
| C. Protected classification characteristics under California or federal law | Gender and date of birth | NO |
| D. Commercial information | Transaction information, purchase history, financial details, and payment information | NO |
| E. Biometric information | Fingerprints and voiceprints | NO |
| F. Internet or other similar network activity | Browsing history, search history, online behavior, interest data, and interactions with our and other websites, applications, systems, and advertisements | NO |
| G. Geolocation data | Device location | |
| H. Audio, electronic, visual, thermal, olfactory, or similar information | Images and audio, video or call recordings created in connection with our business activities | NO |
| I. Professional or employment-related information | Business contact details in order to provide you our Services at a business level or job title, work history, and professional qualifications if you apply for a job with us | NO |
| J. Education Information | Student records and directory information | NO |
| K. Inferences drawn from other personal information | Inferences drawn from any of the collected personal information listed above to create a profile or summary about, for example, an individual’s preferences and characteristics | NO |
| L. Sensitive Personal Information | NO |
Velocity
•
April 1, 2024
Missing dimensions in development ops & productivity metrics
Engineering Managers have a variety of metrics they can draw from to assist in understanding their teams' efficiency & operational performance. What’s missing is a precise measurement of risk (i.e. complexity) and Resilience (i.e. mitigation effort).
Read More
Velocity
•
July 12, 2023
Why it’s time for devs to focus on AI bug detection
From ChatGPT to GitHub Copilot, generative AI is here, and with it has come scores of new technology that’s making it easier than ever for developers to complete coding tasks faster, innovate, and create better software.
Read More
