We have covered the overview of model performance in your previous articles on Einstein Discovery. In this blog, let us dive into learning How to Evaluate a model in Einstein Discovery.
Models, Variables, and Observations
- Models: Einstein Discovery generates and refines a model based on the provided data. This model serves as the foundation for generating diagnostic and comparative insights. Once deployed into production, the model becomes instrumental in deriving predictions and suggesting enhancements for real-time data, a topic we’ll delve into further in subsequent discussions.
- Variables: A variable represents a distinct category of data, akin to a column in a CRM Analytics dataset or a field in a Salesforce object. Within a model, there are two types of variables: inputs, also known as predictor variables, and outputs, which represent predictions.
- Observations: Predictions are made at the observation level, where each observation comprises a structured collection of data. This structure is akin to a populated row in a CRM Analytics dataset or a record in a Salesforce object.
For every observation, there will be a specific set of predictor variables as input , and a corresponding prediction as output. In this example, the Predicted outcome is Converted True, the actual outcome yet unknown.
Prediction Examination
- Click the Prediction Examination tab.
- The Einstein Prediction panel on the right compares the predicted outcome with the actual outcome for the selected row in the training data, alongside highlighting the top factors influencing the predicted outcome. Clicking any row updates this panel.
- This interface offers a valuable preview of how the model will predict outcomes post-deployment. While AUC provides an overall model measure, this screen enables interactive analysis and detailed examination of model predictions.
Explore Predictions and Improvements for a given scenario
Now we can delve into forecasting future outcomes and suggesting enhancements for a given scenario using Einstein Discovery’s predictive capabilities.
- To initiate this process, navigate to the “Predictions” section on the left-hand navigation panel.
- Under “Select a Group to Predict,” choose “Partner Referral” for Lead Source and “Home loan” for Loan purpose.
- Following this, click on the “Actionable” button adjacent to “Lead Source” to unveil actionable insights and improvement suggestions.
On the main page, you’ll find several panels providing crucial insights:
- Einstein Prediction: This panel displays the prediction score based on your selections. For instance, if the predicted outcome is “Converted: True,” it signifies a favorable projection.
- Top Improvements: Here, you’ll discover suggested actions to enhance the predicted outcome. For instance, altering the lead source to “partner referral” could improve the predicted outcome by 0.031, as depicted in the example.
- Model Overview: This section offers quality metrics for your model, providing an overview of its performance and reliability.
- Top Prediction Factors: Explore the most influential variables that impact the predicted outcome. For example, if “Lead Source” is identified as a significant factor and selecting “Partner Referral” improves the predicted outcome by 23.92%, it underscores the importance of this variable in influencing outcomes.
- Insights: Uncover additional insights associated with your selections, providing further context and understanding to inform decision-making processes effectively.
To learn more about models, click About Models (salesforce.com)
Take Five Consulting is a technology company, based in Virginia U.S., that specializes in the Mortgage Banking vertical especially LOS implementation and application development. Take Five Consulting creates and implement mortgage technology and software specifically for Mortgage Industry.
