6 minute read

How to Create Successful Artificial Intelligence (AI) Models

Artificial intelligence has gone from buzzword to mainstream for marketers as it not only allows brands to make sense of large amounts of data in real-time, but because it also frees up time for employees to focus on business priorities instead of spending time on manual processes such as personalization, lead scoring, anomaly detection, and more. According to Gartner, AI embedded in analytics and other marketing software will free up more than a third of employees time in marketing organizations by 2022.

But often, data scientists and marketers don’t know where to start. The task of building an AI model for your organization sounds daunting and ambiguous. And it is. That’s why Dean Abbott, Chief Data Scientist at SmarterHQ, authored his book Applied Predictive Analytics: Principles and Techniques for the Professional Data Analyst - to help data analysts learn the basic principles to begin building predictive AI models.

I recently sat down with Dean Abbott to discuss how to put together successful AI models.

Keep reading below for:

  • How to select the right AI model
  • How to measure the accuracy of an AI model
  • The basics of feature engineering
  • The tools for creating an AI model

And with that, let’s get started:

Thanks, Dean, for taking the time to share with us today the basics of building AI models. Before building an AI model, we first have to ask ourselves - how do you even select the right model?

Great question and exactly where I like to start. The first step in selecting the “right” model is to define what the “right” model should be like! To put this in business terms, what is the business objective of the model? And for that business objective, how can we measure which of the competing models is truly best? The analytics team must work closely with stakeholders to make the metrics of success clear and to translate the modeling objectives the analysts know into business objectives the stakeholders know and understand.

For example, years ago, I was a contractor to the Defense Finance and Accounting Service (DFAS) who wanted to build predictive models to identify invoices that were suspicious and worthy of investigation. Of the more than 1 million invoices each month, they could only manually investigate at most 100 per month.

In this situation, what metric should be used to select the best model? Overall classification accuracy? Certainly not. The best model only has to select the top 100 invoices to investigate well; it is irrelevant operationally how accurate the model is for the rest of the population. So the analysts should tune models to optimize the highest predictive scores. If one picks the best overall model (the one that does best on the entire population), one might end up with a model that actually does quite poorly for the top 100 scores!

Makes sense. That brings me to my next question - how do you measure the accuracy of the model?

I always prefer to measure accuracy in the way that is most consistent with the business objective. Sometimes this might be overall percent correct classification. Sometimes it might be any one of a number of technical metrics that data scientists use, including precision/recall, type I/type II errors, R-squared, and many many more. In my consulting career, most of the models I have built have been related to customer analytics and fraud detection, where there is a “treatment” or “action” is applied to a portion of the entire population, It might be a churn model that will result in a call or email to the high-churn candidates (a small subset of the overall population), or a fraud model that identifies tax returns or credit card transactions that are suspicious and need to be investigated. In these situations, the metric should be one that can identify a subset of the population, such as lift, gains, or ROC area under the curve.

But these are still technical metrics. What should I report to the stakeholders? As much as possible, I like to communicate to them revenue that is expected to be generated by the model (for customer acquisition), or money that is expected to be saved by using the model (churn mitigation models), or recovered revenue from tax models finding non-compliant tax returns, or even how many new customers are expected to have significant revenue after sending them a cross-sell email. Dollars or counts are descriptions that stakeholders can more readily understand than statistical measures.

So we’ve identified how to select the right model and measuring its accuracy. What are the actual tools a data analyst would need to start building the AI model?

Sure. Tools and approaches for building predictive models fall into four categories.

  1. For the self-starters and companies that want to build models themselves, coding the machine learning approaches, including data preparation and model building, can be done in programming languages. The most popular are general purpose languages such as Python and R, database languages like SQL, and for some, analytics languages like SAS and Matlab. Most of these are open source and are therefore free, though some (SAS and Matlab mentioned here) are languages attached to commercial products. The benefit of these languages is that the analyst can customize the analyses in very precise ways, and these languages (especially Python and R) keep very much up-to-date with the latest trends, algorithms, and analysis approaches in the field.
  2. A second approach for self-starters is to use a visual programming interface where the analyst builds off a processing workflow. But rather than using code (text), one uses processing functions built by the software vendor. These are usually much more easily learned than programming languages and are therefore more accessible to non-programmers. Examples include: Tibco KNIME, Rapid Miner, Alteryx, and many more. KNIME is open source for individual users (a commercial version exists for teams that would like more integrated collaboration), Rapid Miner is free for small data sets, but it and Alteryx are most often used as commercial packages.
  3. A third approach is to use a service that builds the models for you, start to finish, like DataRobot and Dataiku. While one can customize analyses with these packages, most users I know begin using them for their automation.
  4. A fourth approach is to use the modeling infrastructure in cloud computing environments like AWS, Google and Microsoft.

As a bonus question for our readers - I frequently see feature engineering as a process one can use for building models. Can you explain more about this process?

Feature engineering is perhaps the most important improvement any modeler can make to the data, and is often touted as THE way to win predictive modeling competitions.

So, what is a feature? A feature is a different version of one or more of the original variables the predictive modeler has available to build models. Some features transform these variables individually into a form that can be used more effectively by the algorithms. Often these features change the shape of the distribution of the original variable so the algorithm won’t be fooled or biased by it. One of the most common examples is monetary variables like salary and home value. Monetary variables almost always have a big tail in its distribution, with larger, unusual values occurring infrequently but severely distorting the average value of the variable. These can be transformed so that the distribution looks like a bell curve (Normal distribution), which many algorithms prefer.

Other features combine multiple variables into a new variable combination that perhaps is impossible for the algorithm to build itself. Examples of multivariate features include ratios (one variable divided by another) and interactions (one variable multiplied by another). These are impossible for many algorithms to discover themselves, and therefore are essential to consider if a predictive modeler wants to find the best model possible.

Thank you so much, Dean, for sharing your knowledge with our readers!