What plays a central role in decentralized AI?
First I discuss briefly what and why, then how in more detail.
What and why
AI is widely accepted as a disruptive technology, a revolution at the level of steam engine, electricity, and Internet. On the other other, economic and societal issues come along. One prominent example is Stack Overflow: from coders’ go-to site for Q&As, to a source of training data for large language models (LLMs), to a steep decline of # questions. Data contributors have not received proper credits, and it is a question how to further improve LLMs for code.
Daron Acemoglu, Simon Johnson, and James Robinson, laureates of 2024 Nobel Prize in economic sciences, “for studies of how institutions are formed and affect prosperity”, discuss the relationship between technology and prosperity, and the influence of institutions on the distribution of power and economic resources, and on incentives and opportunities.
Decentralized AI is a mechanism to foster the AI revolution and to address the economic and societal issues. The synergy of AI and blockchain draws a more concrete blueprint, with details to pan out and to accomplish.
Tokenomics
Tokenomics, or token economics, studies economic aspects of blockchain and cryptocurrency, w.r.t. the design, supply, distribution, valuation of tokens. It is about incentive, motivation, and reward. Several concepts are related: mechanism design, theory of incentives, contract theory, auction theory, and principle-agent theory. Sequential contract design with multiple interactions becomes an emerging research topic. Transaction fee mechanism design is a concrete example.
AI as an asset
All resources related to AI, including talents, data, algorithms, compute, storage, bandwidth, and models, can and should be treated as assets. A contributor should benefit from the contributions, e.g., data markets and token allocation in Bittensor. The advent of large language models (LLMs) raises a new questions: requirements for huge compute. There are decentralized efforts, e.g., Photon for federated LLM pre-training up to 7B parameters.
Blockchain trilemma
Blockchains trade off the security, decentralization and scalability trilemma: consensus security in the presence of malicious players and anti-censorship capability, geographic diversity, connectivity pattern, and synchrony of networked nodes, and security and efficiency with rising transaction throughput and larger network size. Cryptography is critical for security, so are data-driven methods. Decentralization and scalability call for innovations.
Talents around the world are working on them, e.g., Turing laureate Rich Sutton from U of Alberta (the importance of decentralization for intelligence), Turing laureate Shafi Goldwasser and Dawn Song at The Berkeley Center for Responsible Decentralized Intelligence (RDI); Turing laureate Silvio Micali, Christian Catalini and Networked Agents And Decentralized AI (NANDA) at MIT.
Software engineering
Software is eating the world. Software engineering involves design, development, test, and maintenance of software applications. Both AI and blockchain are software. Meanwhile, AI plays important roles in software engineering, with challenges and opportunities, e.g., correctness guarantee. Agents’ fundamental properties of autonomy and optimality are critical for decentralized AI, e.g., for autonomous executions of smart contracts and safety-critical applications like DeFi.
Human behavior
Human behavior is the way people respond to stimuli, mentally, physically, or socially. It is critical for judgement and decision making. Gamification makes enhancements in non-game contexts by simulating game playing experiences for users’ motivation and engagement. Games can change the world, e.g., for better token design and for attracting more people to blockchains.
The commons: tragedy or comedy?
Social dilemmas result from parasitic strategies like free riding and collusion, from the conflict between collective and individual rationality. Worse yet, this may lead to the tragedy of the commons. Elinor Ostrom, the laureate of 2009 Nobel Prize in economic sciences, was positive and talked about the drama of the commons. To overcome such social dilemmas, Ostrom proposed an efficient polycentric governance, through repeated interactions among small, trustful, and communicative groups, which usually outperforms market or government institutions. There are even discussions about the comedy of the commons. Sequential social dilemmas, or common pool resource problems, become an emerging research topic.
Simulation
Simulation is a powerful tool for science and engineering. It is convenient for a simulator to consider complex scenarios, which are beyond analytical approaches, e.g., economics and finance, data-driven mechanism design, and theory of mind.
Tokenomics discussed above follow an analytical approach, in particular, with game theory and mechanism design. Humans are in the loop for AI, blockchain and software engineering. Humans are usually not fully rational, with cognitive and behavioral biases. Moreover, both human and AI agents are usually bounded rational. It is critical to consider advantages and disadvantages of analytical and behavioral approaches and benefit from both, esp. when humans are in the loop. Here comes simulation.
Polycentric governance as proposed by Elinor Ostrom may be the pragmatic, efficient and effective way for further progress of blockchain. The Internet is controlled by centralized authorities. Blockchains are regulated by governments. Popular crypto exchanges like Coinbase and Binance are centralized. Simulation can facilitate the study of polycentric governance of blockchains.
Decision making
Decision making is everywhere, including in decentralized AI, as discussed above. Many decision makings take multiple steps. Reinforcement learning is a natural framework for sequential decision making. Andrew Barto (U of Massachusetts, Amherst) and Richard Sutton (U of Alberta, Amii) won the 2024 Turing Award "for developing the conceptual and algorithmic foundations of reinforcement learning".
It is worthwhile to differentiate between decision making and prediction. Supervised learning, like classification and regression, is for prediction. Note however, imitation learning methods like behaviour cloning and learning from demonstration are supervised learning to approximate reinforcement learning for decision making. Next token prediction in generative pre-trained transformer (GPT), the popular technique underlying most large language models (LLMs), is imitation learning with limitations.
AlphaGo and AlphaZero are very successful, super-human reinforcement learning applications, with wide applications: AlphaTensor for matrix multiplication, AlphaDev for sorting, AlphaChip for chip design, and AlphaProof for maths proving. AlphaEvolve unifies these for designing advanced algorithms. Straightforward applications would be for cryptographic and all sorts of algorithms, from consensus protocols, token design, decentralized computing, to smart contracts, multi-chain inter-operation, all the way to applications like DeFi, RWA, etc.
Reinforcement learning has been applied to sequential contract design, selective data acquisition, and widely in software engineering. Sequential social dilemmas are usually dealt with multi-agent reinforcement learning. With simulators, reinforcement learning can be used to find optimal strategies, e.g., BFTBrain for adaptive Byzantine fault tolerance consensus, AI Economist for taxation policy design, and social environment design for government and economic policy making.
Epilogue
Decision making plays a central role in decentralized AI. Reinforcement learning is a natural framework for sequential decision making, and will make better decisions together with neuro-symbolic, evolutionary, Bayesian, and other AI methods. Learning from experience, trial and error, and iterative improvements are essential to reinforcement learning, decision making and intelligence in general. This will be a big paradigm shift. There are challenges and opportunities, in particular, for decentralized AI. How to collect experience data and how to design efficient learning algorithms, esp. with limited data and resources, would be the next frontier research and business questions. Welcome to the Era of Experience.
