This blog is a continuation of the Building AI Leadership Brain Trust Blog Series which targets board directors and CEO’s to accelerate their duty of care to develop stronger skills and competencies in AI in order to ensure their AI programs achieve sustaining results.
In this blog series, I have identified forty skill domains in an AI Leadership Brain Trust Framework to guide board directors and CEO’s to ensure they can develop and accelerate their investments in successful AI initiatives. You can see the full roster of the forty leadership Brain Trust skills in my first blog.
Each of the blogs in this series explores either a group of skills or does a deep dive into one of the skill areas. I have come to the conclusion that to unlock the last mile of AI value realization that board directors and CEOs must accelerate building a unified brain trust (a unified set of leadership skills that are hardwired in relevant digital and AI skills) to modernize their organizations more rapidly.
- Knowledge is key and if you locked up a room of board directors and CEOs in a board room and asked them
- What steps are required to build a successful AI Strategic Plan and Journey Roadmap – what do you think would be the outcome?
- Where are your AI Investments and have you inventoried them or audited them?
- What is the difference between a computing scientist, a data scientist, and an AI scientist – would their digital literacy skills be sufficient enough to lead and guide their organizations forward?
Sadly, I think we would find some very serious operational execution gaps.
Thus far in this series, we have addressed 10 Strategic Skills, 10 Business Skills, 10 Emotional and Social Intelligence Skills. The next series of blogs will breakdown the Technical skills required for attracting, developing and retaining technical skills in an AI or Data Sciences team.
Below is a summary list of the ten technical skills required, and in this blog, we will discuss the importance of Research Methods Literacy in-depth as in my AI operational experiences, the quality of the research methods, and inspection processes, and investment in AI machine learning operating practices is a major gap across the majority of companies. Without improvements in this skill area, we will continue to have limited success in moving our AI models into sustainable operating practices.
Technical Skills:
Research Methods Literacy
Agile Methods Literacy
User-Centric Design Literacy
Data Analytics Literacy
Digital Literacy (Cloud, SaaS, Computers, etc.)
Mathematics Literacy
Statistics Literacy
Sciences (Computing Science, Complexity Science, Physics) Literacy
Artificial Intelligence (AI) and Machine Learning (ML) Literacy
Sustainability Literacy
Research Methods Literacy
One of the most important skills in being a world-class AI Data Scientist is valuing research rigor to build robust and trusted AI models. Research methods are the strategies, processes, or techniques utilized in the collection of data or evidence for analysis in order to uncover new information or create a better understanding of a topic. There are different types of AI research methods that use different tools for data collection, data preparation, and data modelling.
One of the major concerns in AI design and development is not having the operational processes tightly defined with key review /inspection/quality assurance gates with external reviewers outside the core AI design team. Although it is relatively easy to upload a data set and apply an off-shelf AI algorithm against it and look at the results in a toolkit like Tensor flow, (Google Toolkit), this does not mean the approach /design being used in the research methods are robust, trusted with limited risks.
Executives need to ensure that their AI and Data Science teams have access to quality experimental design research methods.
First What Is Experimental Design? Experimental design refers to how data is allocated to the different groups in an experiment. Types of design include repeated measures, independent groups, and matched pairs designs. An AI or Data Scientist researcher must decide how he/she will allocate their sample to the different experimental groups. Experimental design means creating a set of procedures to test a hypothesis and it is very important that documentation be created along with the experimental design working steps.
What are the required steps to ensure your organization has skilled research methods documented in your AI and Data Science projects?
Here is a list based on my experiences in designing and building complex AI models for numerous customers in diverse industries:
Step One – Define your Research Question – or What is the Big Problem/Challenge you need to solve? Ensure your research question is written in clear and succinct business terms so it is understood. Be crisp and clear to define the benefits/value of solving this problem/challenge and what it would mean to your organization if you solved this problem/challenge? Anchoring your organization’s use case in a customer use case is always a good entry way for funding AI projects, especially if you are in the early stages of AI experimentation. Be tangible in your return on investment (ROI) and cost savings assumptions as much as you can and bring in a finance expert to help validate your business logic. Nothing is worse that the CFO pulling apart your logic in an executive review. Include competitive intelligence if you can to add more value to support your research direction. Remember that you are also selling key stakeholders at this stage, and your goal is to secure our project( order) builds confidence to invest in solving this AI use case/problem. Executive sponsorship is critical in this research alignment step, so ensure this is not taken lightly as “people support what they co-create.”
Step Two – Write down your research hypothesis to guide your research experiment. Writing a good hypothesis posits an expected relationship between the research variables (data) and clearly states a relationship between the variables (data). A strong research hypothesis should be brief and to the point. The research hypothesis to describe the relationship between research variables (data) and should be as direct and explicit as possible. Some of the key attributes of a good research hypothesis are: a) it should be empirical and susceptible to observation. One of the risks in black box AI is not being able to understand /interpret the results increasing risks to observations for humans to understand. b.) a good hypothesis explains a general phenomenon, rather than a single occurrence , often referred to as a generality c.) a good hypothesis should be plausible and not defy logic d.) a good hypothesis is specific and ensures that concepts are carefully and clearly defined (in other words if another researcher picked up the documentation they would have sufficient knowledge to continue with the research experimentation). and 4.) finally, a good hypothesis must be testable (before and after /pre and post validation).
Step Three – Identify your experiment methods/treatments to guide your research. This is one of the most important skills of an AI /data scientist researcher as understanding the different methods and applications of AI and where they are most valuable in solving specific use cases, as well as the types of algorithms best suited for different data set sizes/types will enable more robust model development. Often the most efficient AI methods selected are a combination of machine learning (ML) methods. Supervised and unsupervised machine learning methods, both have their value in solving different problems. What is key is that an AI data scientist must understand the differences and how and when to best apply each method to achieve the most effective results. It is not uncommon that a company selects one type of AI method, without exploring other methods and then in turn, they build sub-optimal AI models that could have been easily avoided had more research experimentation and third-party reviews been involved earlier in the research and design process to mitigate more risks. Providing ample time for research and discovery is key for AI data scientists to do their jobs effectively and efficiently, which is often contrarian to the corporate world where speed has become an activity mantra or frenzy in many North American cultures, where as in China, the vision is often over ten years out and their patience as a culture for the long-term may be their strongest AI skill competency they have in the war for AI IP to move to sustainable outcomes.
Step Four – Gather your data and ensure it is cleansed and does not have data bias. Preparing your organization to ensure it has robust data management practices is a cornerstone for AI and Data Science teams to function efficiently and effectively. Industry research estimates that AI data scientists spend between sixty and eighty percent of their time on data preparation requirements. This enormous time waste, in many respects, results in leaving insufficient time to create, train and evaluate models, let alone have the time to move models into production. The challenge is that most organizations are simply not set up for their data scientists, and other non-IT roles to engage in advanced data analytics. Data assets are often not easily accessible and often the integrated machine learning software toolkits are also not easily accessible, impacting sustainability and success odds. The old world of relying on traditional ETL methods to keep up with real-time demands is no longer realistic and hence moving all data to the cloud and investing in advanced machine learning software tools is a business imperative to enable successful AI value realization outcome. Three best practices that can help guide senior executives to ensure they accelerate their data preparation operating practices are:
- Automate data processes to easily enable data preparation needs. For example, invest in automatic gender identification in data sets, easy data labelling toolkits, data preparation tools to easily match data or delete data duplications, screen for data bias, or insufficient data to preform sufficient testing. Ensure your data preparation toolkits enable, strong filtering, profiling, feature identification, searching as well as structuring data and model preparation (transformations).
- Invest in machine learning software that supports a self-service collaborative environment to help support cross-functional work team, share plans, insights and improve knowledge flow/and share-ability of diverse data pipelines to expedite/validate data preparation and data sign-off /review practices.
- Migrate to the Cloud more rapidly – to enable AI data access – this means moving strategic data sources to the cloud for increased ease of access for analytics and decision making, otherwise, executives must be more more patient to deal with complicated on premise data stores that are not in data lakes or cloud infrastructures. Being realistic on the excessive time in data preparation methods is a sobering reality but executives must leapfrog ahead in this area if they ever hope to have a world-class AI operating infrastructure for advanced analytics.
Step Five – Select your AI method(s) to execute your research experiment. Once you have your data prepared, you need to think of the appropriate AI method(s) to perform on your data set. There are many types of AI analytical techniques, for example: Heuristics, Support Vector Machines, Neural Networks, the Markov Decision Process, and Natural Language Processing (NLP) are all types of AI analytical approaches that executives need to understand to increase their knowledge of AI. A handy reference guide which explains these methods in simple terms prepared by Deloitte can be found here.
Step Six – Analyze, interpret and present the research results – This skill area is arguably the most important communication building black where the hypothesis comes to life with clearly summarized observations that provide fact based insights to support the problem being solved. Its critical in this step to sequence and record all the model versions/model methods/approaches that were taken to demonstrate the rigour in the analytical approach in finding the most optimal model to predict reliably trusted outcomes that decisions can be made from. In this area, often the data scientists need support from business strategy experts and communication experts to write up the findings that will resonate with senior executives in business language and value realization outcomes that they can appreciate, understand and align with. A data scientist often lacks business communication writing skills, so shoring up the required support skills is key to ensure knowledge is not lost.
In summary, ensuring your AI and data sciences team are able to develop successful experiments and apply iterative approaches is important. Keeping the research experiments tight and simple will increase confidence in sustaining continued investments into AI modelling experiments. There is nothing better than seeing concepts being tested in executable actions, and building proof of concepts (POCs) to help validate proof to run future experiments. What is also key from the get go is thinking big so AI experiments can follow a visionary outlook to keep the confidence growing and monitoring different experiment’s evolution in advancing the longer term goals and vision of the organization.
Board Directors and CEO’s are usually not skilled in AI research methods and I recommend that they upgrade their knowledge as AI is all about quality research experimentation. This means that companies must be focused on developing a strong data culture and investing in modernized data management toolkits to ensure that their organizations can secure the results from their AI programs.
With few companies successfully moving AI models into sustainable operating practices, there is even more reason for board directors and CEO’s to request a third party review of their AI and Data Science operating practices to help guide their organizations foreward. Like board directors and CEO’s had to learn about the importance of Supply Chain Management (SCM) operating practices and Customer Relationship Management (CRM) operating practices, the time is here for them to accelerate their knowledge of AI Analytical Management Practices (AIAMP) and Machine Learning Practices (MLOps). AI is a skill that every C level must advance to learn on some level, it requires increased digital and analytical literacy investments . It also requires a relentless passion /recognition/understanding that a strong data culture, where data is viewed as a strategic asset is a responsibility of every employee will take time, but well seeded, it will help to ensure your company’s future is modernized and quite frankly, survives.
The AI tidal wave is now here and its washing rapidly over all business processes, what we now need is to ensure our board of directors and CEO’s are not washed up or beached, due to their lack of learning curiosity or appreciation of the magnitude of this transformational change in play.
originally posted on forbes.com by Cindy Gordon
About Author: Dr. Cindy Gordon is a CEO, a thought leader, author, keynote speaker, board director, and advisor to companies and governments striving to modernize their business operations, with advanced AI methods. She is the CEO and Founder of SalesChoice, an AI SaaS company focused on Improving Sales Revenue Inefficiencies and Ending Revenue Uncertainty. A former Accenture, Xerox, and Citicorp executive, she bridges governance, strategy, and operations in her AI contributions. She is a board advisor of the Forbes School of Business and Technology and the AI Forum. She is passionate about modernizing innovation with disruptive technologies (SaaS/Cloud, Smart Apps, AI, IoT, Robots), with 13 books in the market, with the 14th on The AI Split: A Perfect World or a Perfect Storm to be released shortly. Follow her on Linked In or on Twitter or her Website. You can also access her at The AI Directory.
