Documenting potential biases and diagnosing where they arise supports fairness and ethics goals for machine learning models by making the decisions and limitations of the algorithm more transparent. The following structured approach to document potential biases can be applied:

• Document preprocessing steps that may inadvertently introduce biases (e.g., normalization, imputation techniques)
• Identify features that may be proxies for sensitive attributes (e.g., gender, race) and prone to introducing biases. Document decisions related to the inclusion or exclusion of such features in the model.
• Document the choice of algorithms and their potential biases (e.g., models trained on biased datasets, biased loss functions) and note any hyperparameter settings that may amplify or mitigate biases.
• Document efforts to address class imbalances and potential impacts on model fairness and note any techniques used to mitigate biases during training.
• Document the choice of evaluation metrics, consider whether they are appropriate for all relevant groups, and be aware of metrics that may mask disparities or unfairly advantage certain groups.
• Document potential biases that may arise in the deployment environment (e.g., changes in user behavior, evolving data distributions). Consider any biases introduced during the deployment of models in specific applications
• Document biases that may arise in user feedback and consider their potential impact on model improvement efforts
• Document how the model aligns with ethical guidelines and principles established by the organization and identify ethical concerns and potential societal impacts.

‍

Refer to the Risk and Fairness section for more information about bias documentation. 

‍

Feature selection involves isolating the most consistent, non-redundant, and relevant features to use in model construction.² Transparent feature selection processes enable users and data scientists to validate the model's inputs and challenge any unjust or discriminatory patterns that may emerge. Some feature importance techniques are:

• Permutation Importance: Assess the impact of shuffling individual features on model performance. If shuffling a feature significantly reduces model performance, that feature is considered important.
• Feature Importance from Tree-Based Models: Tree-based models (e.g., decision trees, random forests, gradient boosting) naturally provide feature importance scores based on the contribution of each feature to the model's splits.
• LASSO (L1 Regularization) Coefficients: In linear models with L1 regularization, the magnitude of the coefficients represents feature importance. Non-zero coefficients indicate important features.
• Recursive Feature Elimination (RFE): This method iteratively removes the least important features until the desired number is reached, providing a feature importance ranking.

‍

In machine learning models, optimal feature selection produces greater predictive and explainability power. Understanding and communicating feature importance contribute to model transparency, interpretability, and trust, allowing stakeholders to make informed decisions based on the model's insights. More information about feature selection can be found in the section of Considerations along the ML pipeline: In-Processing.

‍

The online book "Interpretable Machine Learning" by Christoph Molnar is a valuable resource for gaining insights into explainable machine learning.

‍

‍

The documentation of the characteristics of the training dataset can uncover potential biases and help stakeholders, including data scientists, policymakers, and users, gain insights into the representation of different demographic groups in the data. Demographic representativeness (see the Stakeholder Engagement section for more details) in training data and model transparency is necessary to ensure that machine learning models are deployed in an equitable and reproducible manner. Wider adoption of reporting guidelines is warranted to improve representativeness and reproducibility.³ 

‍

Document the following attributes of the training dataset:⁴

‍

Demographic Composition: Composition of relevant demographic characteristics such as age, gender, race and ethnicity, geographic location, income level, and education level.

‍

Data Collection Methods:

• Sampling Strategies: Document the methods used for sampling data, including any oversampling or undersampling techniques.

• Data Sources: Specify the data sources, such as surveys, public records, or online platforms, and assess the representativeness of these sources.

‍‍

Potential Biases in Data Collection:

• Underrepresentation: Identify any groups that may be underrepresented in the dataset and consider the implications for model generalization.⁵
• Sampling Bias: Document any biases introduced during the sampling process and assess their impact on the training data.

‍

Imbalances and Disparities:

• Class Imbalances: Note if there are imbalances in the distribution of demographic classes, especially this is relevant for classification tasks.

• Disparities: Assess if there are disparities in the quantity of data available for different demographic groups.

‍

Data Privacy and Ethics:

• Privacy Considerations: Ensure compliance with privacy regulations and document any steps taken to protect sensitive demographic information.

• Ethical Considerations: Document ethical considerations related to using demographic data, particularly when dealing with vulnerable or marginalized groups.

‍

Developers should adopt the following practices as they work with the dataset to ensure clear documentation as it evolves:

‍

1. Documentation Format:

• Metadata: Include metadata that summarizes the training dataset's demographic composition.
• Visualizations: Use visualization techniques such as bar charts, pie charts, or heatmaps to illustrate the distribution of demographics.
• Textual Description: Provide a written description of key findings and considerations related to the demographics of the dataset.

‍

2. Adopt Version Control: Implement version control for documentation to track changes  in the dataset, demographics of the dataset, any data augmentation procedures such as generating synthetic data to enhance equitable data representation  over time. Update documentation as new information becomes available.

‍

3. Stakeholder Involvement: Seek input from diverse stakeholders, including ethicists, domain experts, and community representatives, in the discussion and documentation process. Communicate findings and potential biases to relevant parties.

‍

‍

To convey to end users how the model works, developers can follow the guidelines below.⁶ Transparent communication about the workings of a machine learning model is crucial from an equity perspective to build trust and accountability among end users and stakeholders. Developers can achieve this by clearly explaining the model's decision-making process using non-technical language and visual aids, while being transparent about the data used, including biases and limitations. Clearly communicating the model's limitations supports expectation management and establishes feedback mechanisms for users to raise concerns. Empowering users with tools for independent exploration of the model's decisions, discussing ethical considerations, and offering continuous education on machine learning and data science concepts further promote transparency, trust, and fairness in model deployment and usage.

‍

Document the following attributes to document how the machine learning model works:

‍‍

Model Architecture: Briefly describe the type of machine learning model employed (e.g., regression, classification, neural network).

‍

Training Process: Explain how the model was trained using historical data to learn patterns and relationships.

‍

Input Features:

• Identification: List and describe the input features used by the model. Specify which features the model considers as significant for predictions.
• Data Representation: Explain how input data is represented and processed by the model (e.g., encoding, scaling).

‍

Prediction Mechanism: Describe how the model makes predictions based on new input data.

‍

Key Components:

• Feature Importance: If applicable, discuss which features the model considers most important in making predictions.
• Weights and Coefficients: If relevant (e.g., linear models), discuss the weights or coefficients assigned to each feature.

‍

Explainability Techniques:

• Model Interpretability: Describe any techniques used to enhance the interpretability of the model (e.g., SHAP values, LIME).
• Visualization Tools: If applicable, mention any visualizations or tools used to explain the model's decisions.

‍

Learning from Data: Explain whether the model is capable of learning and adapting to changes in input data.

‍

Limitations and Assumptions:

• Domain Constraints: Highlight any limitations or constraints within the problem domain the model may encounter.
• Assumptions: Clearly state the model's assumptions during the prediction process.

‍

Ethical Considerations:

• Bias Mitigation: Describe measures taken to mitigate biases in the model's predictions.
• Fairness and Accountability: Address ethical considerations related to fairness and accountability in model predictions.
• Feedback Loops: Explain how user feedback is incorporated to improve model performance.
• User-Friendly Outputs: Detail efforts made to present model outputs in a user-friendly and understandable manner.
  ‍

Continuous Improvement: Explain how the model's performance is monitored and outline plans for iterative improvements and updates based on ongoing evaluations.

‍

‍

An AI product is only as good as its usability and the ease with which users can interact with it.⁷

‍

Developers can use the following guidelines to ease users' adoption:

Overview of the Product: Start by providing an overview of the AI product, including its purpose, main features, and capabilities. This helps users understand what the product does and how it can help them.

‍

User Scenarios: Provide specific examples or scenarios that illustrate how users can use the product in their daily lives or work. This helps users visualize how the product can be applied to real-world situations and solve their problems. These scenarios should include:

• Context about the user's situation or environment. This helps users understand the scenario and relate it to their own experiences.
• A clearly articulated problem statement that explains the challenge that the user is facing. This helps users understand the motivation behind using the AI product and how it can help address their needs.
• The goals or objectives that the user wants to achieve. This helps users understand what they are trying to accomplish with the AI product and how it fits into their workflow or activities.
• An explanation how the user interacts with the AI product to achieve their goals. 
• The benefits or outcomes that the user experiences as a result of using the AI product. 
• A description of the end result or resolution of the scenario. This helps users understand the impact of using the AI product and how it contributes to their overall success.

User Onboarding: Provide a user-friendly setup process, guiding users through the initial steps to use the AI product. Offer tutorials or guides explaining key features and functionalities and provide customer support for end users.

‍

Input Data Preparation: Specify the required format for input data, ensuring users understand how to structure the data. Implement validation mechanisms to catch errors in input data.

‍

Parameter Tuning: Identify and communicate parameters that users can tune based on their needs. Provide default settings but allow customization for diverse use cases.

‍

Feedback Mechanisms: Establish channels for users to provide feedback on the AI product's performance. Encourage users to report issues or suggest improvements. Stay attuned to evolving user needs and adapt the product accordingly.

‍

Monitoring and Analytics: Provide users with dashboards displaying key performance metrics. Enable users to monitor the AI product's performance over time.

‍

Adaptability to Data Changes: Implement mechanisms to detect and alert users to changes in the input data distribution. Provide guidance on adapting the model to new data.

‍

Scaling and Resource Management: Offer guidelines for scaling the AI product as user demands grow. Provide recommendations for resource allocation and optimization.

‍

Integration with Existing Systems: Provide clear documentation and APIs for seamless integration with other systems. Ensure compatibility with common data formats and protocols.

‍

Security and Privacy Considerations: Outline security measures in place to protect user data and maintain privacy. Educate users on best practices for secure usage. 

‍

Version Control and Updates: Implement a versioning system for the AI product to manage updates. Communicate changes, improvements, and potential impacts with each version.

‍

The AI product can remain adaptable, user-friendly, and aligned with evolving requirements by providing a user-centric approach and addressing circumstances that may necessitate modifications.

‍

For more guidance and support with stakeholder engagement, explore this helpful resource: Stakeholder Engagement Throughout The Development Lifecycle.

‍

Reference this resource we created, Model Explanation Guiding Questions to support your discussion at this phase.

‍