Courtesy of Joe Glick, Chief Data Scientist, RYLTI
Volatility refers to rapid and unpredictable change. Since the risks are unpredictable, how can they be managed? There are three areas where human-led governance that is enabled by biomimetic knowledge engineering methodology and software (digital twins) can optimize our risk management process.
- Before AI/ML models are designed
- Before model outputs are integrated with downstream processes
- Monitoring the results of utilizing the outputs
Before Design – Twin the Problem Definition
Premises driving the approach, and theories guiding the architecture need to be identified, modeled, and validated. One twin is required for premises and another for theories, and the modeling process helps to distinguish between expected and proven knowledge, supporting human-led governance. Although the usage of the terms varies, from a scientific viewpoint:
- Theories are preliminary conclusions based on observed evidence, but not yet tested
- Premises are assumptions in search of evidence, and frequently vague, for example: “In a lot of neuroscience, the premises remain unexamined, but everything else is impeccable. “- John Krakauer, neuroscientist at Johns Hopkins University
Similarly, impeccable algorithms that implement unexamined premises fuel unpredictable risks and are a key source of bias. Likewise, theories that are not clearly defined and mapped to existing and expected evidence are difficult to validate. This initial phase of human-led governance is critical to mitigate significant AI risk, and it enables the next two phases.
Before Integration – Twin the Solution Methods
Usually, the final specification of the software is delivered once the coding is done. Each implemented solution needs to be modeled as an independent agent in a Solution Methods twin that can interact stochastically with the Premises and Theories twins. The three twins create an ecosystem to validate the new software by outputting various scenarios that are reviewed by relevant experts.
Monitoring Results – Test Using Problem and Solution Twins
Once the software is in Beta testing, all the outputs need to be delivered to a data lake accessible to the digital twin ecosystem. A fourth twin, Beta Discovery, explores the outputs for dark data (categories not specified in the solution schema and unexpected correlations) and interacts with the Problem and Solution twins to produce evidence of quality, correctness, and bias, enabling experts to manage volatility risk.
The biomimetic digital twin ecosystem is then leveraged and adapted to address the remaining VUCA drivers.