Google Proposes TabFM, a Foundation Model for Tabular Data

Google Research has proposed TabFM, a new foundation model specialized for tabular data. Rather than training a model from scratch for each individual dataset as in the past, it adopts an architecture capable of generating predictions for previously unseen tables through a single inference pass.

The majority of business data is managed in tabular format such as data warehouses, customer relationship management systems, and financial ledgers. However, to create predictive models from such data, complex preparatory work has traditionally been necessary, including data preprocessing, missing value imputation, numerical conversion of categorical variables, and feature engineering. Moreover, even after training, hyperparameter optimization must be repeated and retraining pipelines must be run continuously whenever data distribution changes, incurring ongoing operational costs. Weihao Kong, a research scientist at Google Research, characterizes this situation as creating "continuous operational debt in maintaining adaptation to data changes."

In response to these challenges, the generative AI field dealing with text and images has already adopted "zero-shot inference," a technique that accomplishes new tasks with only a prompt and without additional training. It naturally follows that one might simply feed tables directly into existing large language models (LLMs). However, because LLMs are trained on natural language, they struggle with structured data processing. With just thousands of rows and hundreds of columns, context limits are exceeded, numerical tokenization reduces accuracy, and converting two-dimensional tables into one-dimensional text causes loss of row-column correspondence. Kong also notes that "today it is far more effective to have an LLM write code to call XGBoost than to directly feed tables to an LLM."

TabFM attempts to solve this problem using a technique called "in-context learning." In-context learning is a mechanism where, without updating (retraining) model weights, known data and new data to be predicted are provided together as a single prompt, and the model itself interprets the relationship between columns and rows at runtime to produce predictions. In other words, there is no need to retrain every time a new table arrives, and inference is completed in a single "forward pass."

The practical implications are significant. With traditional approaches, work from data pipeline construction to production deployment required weeks of effort. Within the TabFM framework, this can be shortened in principle to simply calling an API once. For corporate developers and AI engineers, this offers potential to substantially compress the lead time for deploying predictive models.

The direction shown by TabFM is noteworthy in that it reconsiders the operational model of machine learning itself. Traditionally, tabular data prediction has assumed extensive tuning by data scientists, but if foundation models can assume part of that role, the scope for AI adoption is likely to broaden. However, accuracy on actual business data and the ability to handle large-scale tables remain at a stage requiring further validation. Whether TabFM reaches a level capable of withstanding industrial application will be the next point of focus.