May 10, 2023

Why Software Engineers Need Better Risk Assessment Tools for Pull Requests

by Mark Greene


As a software engineer, you may have shipped something that you and the reviewers knew was risky, but it was too expensive to investigate the issue further. You may have added remediation steps, but time pressure may have prevented you from doing so.

Software Engineers are inherently risk managers but often not thought of as having this responsibility. After their work is deployed however, they are expected to respond and diagnose production issues. Perhaps this is one reason why the observability tooling are so robust. It’s time to shine a light on this underemphasized yet critical job responsibility.

Having objective AI-powered tools to aid us can lead to positive behavioral change not only for engineering but for the whole organization. Shepherdly provides a predictive model that is trained on your production error and bug data. This is important because it aligns risk with customer pain, a concept the whole organization can align around and empathize with.

Here are a few ways to apply a risk assessment tool like Shepherdly to your workflow:

JIT Remediation

Know when to deploy your de-risking efforts and to what degree.

It’s common for teams to have existing playbooks for guarding against risk. The challenge is applying them with precision. Risk is a spectrum and so should the remediation tactics. Shepherdly’s risk score acknowledges this fact and provides a range of 1-100 along with the statistical probability of causing a bug. While this is team specific, a corresponding spectrum of remediation could look something like:

  • Extra reviewers with a focus on bug inspection
  • Expanded automated test coverage
  • Feature Flags
  • Increased observability (tracing, logging, product usage metrics) for changed code
  • Dedicated QA investment (if staffed)
  • Modified phased rollouts

Because Shepherdly uses the probability of a bug being introduced as its proxy for risk, paired with training on your actual production data, the risk scores are personalized to your environment and more importantly, the impact your customers feel.

Customer Driven Tech Debt

Identify & prioritize tech debt by actual business value, with objective data aligned with your customers.

One of the challenges in managing technical debt is prioritizing it effectively. Simply showing a product manager the cyclomatic complexity of your code base may not be enough to convince them. However, demonstrating that your team’s repository has the most customer-facing bugs can quickly change the conversation.

Within a repository, teams can gain precise insight into which areas of the codebase to target for refactoring by identifying the files that are the most fragile.

Sprint Planning

Understand how tech debt is going to skew your estimate.

During the scoping & estimating process, it can be tricky to convey an anticipated area of the code that will be difficult to modify. With the Hotspot view in Shepherdly, you can quickly identify if an area of the codebase you’ll be modifying is brittle and therefore will require more effort to work through in order to reduce bugs and minimize the impact of bugs occurring in production.

Post navigation

Next Post →

Table Of Contents

A. IdentifiersContact details, such as real name, alias, postal address, telephone or mobile contact number, unique personal identifier, online identifier, Internet Protocol address, email address, and account nameYES
B. Personal information categories listed in the California Customer Records statuteName, contact information, education, employment, employment history, and financial informationNO
C. Protected classification characteristics under California or federal lawGender and date of birthNO
D. Commercial informationTransaction information, purchase history, financial details, and payment informationNO
E. Biometric informationFingerprints and voiceprintsNO
F. Internet or other similar network activityBrowsing history, search history, online behavior, interest data, and interactions with our and other websites, applications, systems, and advertisementsNO
G. Geolocation dataDevice location
H. Audio, electronic, visual, thermal, olfactory, or similar informationImages and audio, video or call recordings created in connection with our business activitiesNO
I. Professional or employment-related informationBusiness contact details in order to provide you our Services at a business level or job title, work history, and professional qualifications if you apply for a job with usNO
J. Education InformationStudent records and directory informationNO
K. Inferences drawn from other personal informationInferences drawn from any of the collected personal information listed above to create a profile or summary about, for example, an individual’s preferences and characteristicsNO
L. Sensitive Personal InformationNO