June 23, 2023

AutoReviewer: Bug History-Guided Code Review

by Mark Greene


PR Reviews Just Got a Lot More Effective

In addition to predicting the likelihood of a PR causing a bug, Shepherdly adds even more context to the bug hotspots in your code when modified. AutoReviewer will be an essential assistant for engineers this year as code velocity increases, fueled by LLM-driven dev tools. As the pace of coding accelerates, so does the potential for bugs – that’s where AutoReviewer steps in as a critical tool to mitigate this risk.

History and Context, Delivered Directly in Your PR Flow

AutoReviewer goes beyond just classifying bugs. By analyzing your project’s bug fix history, it provides developers with essential context about errors within the codebase. This information is instrumental in understanding how bugs have manifested in the past, their impact, and the approaches that resolved them.

Quantifying Bug Magnitude for High-Risk Files

An exciting feature of the AutoReviewer is its ability to embed the magnitude of bug fixes for each high-risk file. Used in conjunction with the Risk Score, this provides developers with a comprehensive and contextual understanding of a file’s fragility. This insight helps justify expensive remediation tactics like phased rollouts or implementing more extensive automated testing procedures like integration tests or engaging a QA team.

A PR with a high risk score and a significant bug fix ratio strongly signals that any changes should be deployed with thorough remediation and caution.

High-Level Error Themes and Actionable Suggestions

More than just an error reporting tool, AutoReviewer generates high-level themes for each bug occurrence per file, providing valuable insight into recurring issues and their root causes. This summary gives developers months or even years of bug fix patterns right in the PR flow. If the lines changed bear similarities to the bug fix history, the system provides targeted suggestions to help focus attention where it’s most needed.

How Effective This Can Be

A study by Braz et al., titled “Less is More: Supporting Developers in Vulnerability Detection during Code Review” highlights that merely instructing reviewers to prioritize security during code review can boost vulnerability detection by eightfold. This finding implies that if you give engineers a solid reason to concentrate on a specific area, their effectiveness in bug detection increases dramatically.

Table Of Contents

A. IdentifiersContact details, such as real name, alias, postal address, telephone or mobile contact number, unique personal identifier, online identifier, Internet Protocol address, email address, and account nameYES
B. Personal information categories listed in the California Customer Records statuteName, contact information, education, employment, employment history, and financial informationNO
C. Protected classification characteristics under California or federal lawGender and date of birthNO
D. Commercial informationTransaction information, purchase history, financial details, and payment informationNO
E. Biometric informationFingerprints and voiceprintsNO
F. Internet or other similar network activityBrowsing history, search history, online behavior, interest data, and interactions with our and other websites, applications, systems, and advertisementsNO
G. Geolocation dataDevice location
H. Audio, electronic, visual, thermal, olfactory, or similar informationImages and audio, video or call recordings created in connection with our business activitiesNO
I. Professional or employment-related informationBusiness contact details in order to provide you our Services at a business level or job title, work history, and professional qualifications if you apply for a job with usNO
J. Education InformationStudent records and directory informationNO
K. Inferences drawn from other personal informationInferences drawn from any of the collected personal information listed above to create a profile or summary about, for example, an individual’s preferences and characteristicsNO
L. Sensitive Personal InformationNO