Random Thoughts and Observations on Data Science and Beyond

GenAI: Pathways from Art to Science


“Three Pathways, AI-generated Image, Microsoft Copilot, 2024”

For many practitioners (and not just research scientists) the most troubling part about Generative AI (GenAI) is the absence of a robust scientific framework for GenAI. This has been reported in the Computer Vision area as recently as in 2021A.
In the past there have been numerous instances in which the artifact has preceded the development of the theory: the steam engine preceded the formulation of the Laws of Thermodynamics, the telescope preceded Optics,…B.
To my mind there are three pathways by which GenAI foundations will develop.

  1. A grand theory of Large Language Models (LLMs) that describe inviolable rules by which LLMs are governed.
  2. Componentization of the LLM model in which each component is described by inviolable rules.
  3. Componentization of the LLM model in which each component is described by probabilistic rules (like the diffusion process).

All paths are likely to be developed in an iterative way. The third pathway is most likely. In any case what would such (probabilistic or deterministic) rules describe? Or would GenAI continue to stay more of an Art and less of a Science. What do you think?

Note-A: See “The Affective Growth of Computer Vision” by Su and Crandall. https://vision.soic.indiana.edu/papers/affective2021cvpr.pdf (4.2.1 From Scientists to Neural Network Technicians)
Note-B: See Yann LeCun: Turing Award Lecture “The Deep Learning Revolution” (https://youtu.be/psl1wY2V-L0 34 min 50 seconds).

Aniruddha M Godbole is an AI & Data Science practitioner. He is a continuous learner. These are his personal views.