BigQuery AI in pandas: cloud execution, costs and limits
Google’s November 6 walkthrough shows how pandas users can call BigQuery AI functions without rebuilding their notebook around a different data interface. The practical benefit is simpler access to forecasting, tabular prediction, embeddings and semantic filtering. The important boundary is where the work happens: the familiar pandas expression sends computation to Google Cloud.
Importing bigframes.pandas enables the .bigquery accessor on ordinary pandas objects. BigFrames translates supported operations into SQL, executes them through cloud services and returns pandas results. A CSV loaded on a laptop can therefore become input to a remote workflow. Teams should approve that data movement before using confidential datasets.
A November walkthrough, not a new launch
The BigFrames changelog records the pandas accessor in version 2.38.0 on March 16, 2026. The pandas forecasting accessor followed on March 31, with more AI functions added through July. Google’s recent article brings those capabilities together in examples. It does not establish that the accessor or its underlying models first became available in November.
Start with the project and the task
Google’s runnable accessor notebook provides entry points for Colab, Colab Enterprise and BigQuery Studio. Its setup specifies a Google Cloud project and location, with the BigQuery Job User and Agent Platform User roles. Importing a library alone does not supply cloud permissions or choose the right billing project.
The examples separate several jobs. Embedding generation turns a text column into vectors. Similarity compares a series with one phrase or another series. Forecasting takes structured time series, while prediction combines labeled historical rows with separate rows that need an answer. Choose the function around the required output rather than treating every AI call as interchangeable.
TabFM prediction still needs labeled examples
The AI.PREDICT reference identifies TabFM as the pretrained model for regression and classification and marks the feature Preview. It accepts labeled reference data and prediction data with matching feature columns. Avoiding a separate model training pipeline does not mean making useful predictions without examples.
The label’s type determines the task: numeric labels select regression, while strings and booleans select classification. Supported inputs include strings, booleans and the documented numeric types. The current limits are 50 feature columns and ten classification categories. Those constraints should be checked before adapting a wide business dataset or a classifier with many possible outcomes.
Google points to AI.EVALUATE for assessing TabFM performance. For a trial, reserve rows with known answers and compare the predictions with a simple existing baseline. That comparison is more informative than counting how many lines of notebook code disappear. ByteForward has not benchmarked TabFM or verified accuracy on a production dataset.
Semantic filtering has a cost and quality choice
The AI.IF reference describes a natural language condition that returns true or false for each row. Its default cost-oriented mode uses a distilled model for most rows. The quality-oriented mode always calls the remote language model. If no endpoint is supplied, BigQuery chooses one based on the task’s cost and quality tradeoff.
Record the endpoint and optimization setting when comparing runs. A filter for customer requests about a delayed shipment may need a different tolerance for missed matches than a casual exploratory search. Review representative true and false results before using a semantic condition to route work automatically.
A free library can still create a cloud bill
The BigFrames introduction distinguishes the free, open source Python library from the Google Cloud services it uses. Those services have their own charges and quotas. Its general storage guidance also says temporary intermediate tables persist for seven days by default and incur storage costs. That is library-level documentation, not a claim that we observed identical retention for every accessor method.
Budget for query processing, applicable model inference and stored intermediate data. The BigQuery pricing page separates compute and storage and describes additional model charges. Check the selected function and billing arrangement before scaling a notebook. At this review, Google’s TabFM reference and pricing page disagreed on the transition date for token billing, so teams should confirm the current applicable terms rather than rely on a copied start date.
Choose the workflow before expanding it
This accessor fits a developer who wants cloud AI inside a pandas analysis. Employees asking broad questions in Gemini Enterprise use a different route, covered in our guide to Data Cloud connectors, access and query costs. Similar database branding does not make the two workflows equivalent.
Begin with a small approved dataset and one measurable task. Confirm the cloud location, inspect returned types and failures, and record elapsed time, accuracy and actual charges. Keep the shorter interface if it makes the whole workflow easier to operate, not simply because the first example fits in fewer lines.
AI-generated editorial illustration by ByteForward, using Google’s BigQuery product icon. Google Cloud and BigQuery are trademarks of Google LLC. ByteForward is not affiliated with or endorsed by Google.
