Overview: Output Adjustment & Interpretation
Output adjustment and interpretation in machine learning refer to techniques used to modify or interpret model predictions to make them more equity-focused, less biased, meaningful, actionable, and understandable. This section covers the potential impact assessment of the output before human interpretation and contextualization of results using an equity-driven framework.
Potential Impact Assessment of Output
Practical testing of a machine learning model's output before human interpretation is critical to ensure that the model's predictions or classifications are fair, accurate, and trustworthy, especially when decisions based on these outputs may have significant societal impact. This testing phase allows developers to evaluate the model's performance in real-world scenarios and identify any biases, disparities, or unintended consequences that may arise. Three testing methods can be used:1
The invariance test defines input changes that are expected to leave model outputs unaffected. A common method for testing invariance is related to data augmentation. It involves pairing up modified and unmodified input examples and seeing how much this affects the model output.
A directional expectation test is run to check how a change in the input distribution changes the expected output. An example is testing assumptions about the number of bathrooms or property size when predicting house prices. A higher number of bathrooms should mean a higher price prediction. Seeing a different result might reveal wrong assumptions about the relationship between our input and output or the distribution of our dataset.
The minimum functionality test helps developers decide whether individual model components behave as expected. Overall, output-based performance can conceal critical upcoming issues in the model; therefore, developers must run this test to reveal this issue. A few ways to do this are:
- Create samples that are “very easy” for the model to predict to see if they consistently deliver these types of predictions;
- Test data segments and subsets that meet specific criteria;
- Test for failure modes identified during manual error analysis.
Monitor Results and Contextualize Results
Continuous monitoring allows organizations to track the model's performance over time and detect any changes or deviations from expected outcomes. By contextualizing results within the broader socio-cultural context, organizations can identify potential biases, disparities, or unintended consequences that may disproportionately impact marginalized or vulnerable communities. This contextualization helps ensure that the model's outputs are interpreted and acted upon in a manner that upholds equity.2 The following blueprint can be used to guide this stage of the ML development and production process:
Define key performance indicators (KPIs) based on project goals and objectives, such as accuracy, precision, recall, F1 score, or business-specific metrics. Identify additional metrics that provide context to model performance, such as fairness metrics (e.g., disparate impact) or interpretability metrics (e.g., feature importance).
Set baseline values and acceptable ranges or thresholds for KPIs based on business requirements, regulatory constraints, or industry standards; then use them to compare against model performance.
Implement automated processes to monitor model performance in real-time or at regular intervals. Set up alerts or notifications for deviations from expected performance or when thresholds are breached.
Interpret model performance in the context of broader business objectives and strategies. Relate model outcomes to business impact, such as revenue, customer satisfaction, or operational efficiency. Involve domain experts to provide insights and interpret results based on their domain knowledge.
Conduct comparative analyses. Analyze trends in model performance over time to identify patterns or changes. Compare current results with historical data to assess progress or detect anomalies. Benchmark model performance against industry standards, competitor models, or state-of-the-art approaches.
Undertake a root cause analysis. Investigate potential factors contributing to changes or deviations in model performance. Consider data quality issues, changes in the input data distribution, or model drift. Debug model predictions to understand the reasoning behind specific outcomes. Evaluate model behavior under different conditions or edge cases.
Incorporate user feedback and stakeholder input to assess model performance and relevance. Use feedback loops to improve model outcomes based on user experiences iteratively. Translate feedback into actionable improvements or adjustments to the model architecture, features, or training data.
Document monitoring results, contextual analysis, and actions taken in a transparent and accessible manner. Communicate findings to relevant stakeholders through regular reports or dashboards.
By effectively monitoring and contextualizing the results of ML/AI models, organizations can gain deeper insights into model performance, identify areas for improvement, and ensure that machine learning solutions align with organization objectives and user needs over time.
For more guidance and support with stakeholder engagement, explore this helpful resource: Stakeholder Engagement Throughout The Development Lifecycle.
Reference this resource we created, Output Adjustment & Interpretation Guiding Questions, to support your discussion at this phase.
- How to Test Machine Learning Models. (2022). Deepchecks. deepchecks.com
- Cohen, O. (n.d.). A Comprehensive Guide on How to Monitor Your Models in Production. Neptune. neptune.ai