Tags describe technologies and technical subjects. Combine them with platform tracks in the article archive.
Access Control
RLS in Databricks Unity Catalog and Power BI
Linked Service Property Magic
Enhancing Your Lakehouse with Unity Catalog
Azure Synapse / SQL User Provisioning the Right Way
Apache Spark
Mastering Spark: DataFrameWriterV2 vs. DataFrameWriterV1
Mastering Spark: Should You Infer Schema in Production?
Notebooks, Spark Jobs, and the Hidden Cost of Convenience
Creating your first Spark Job Definition
The Small Data Showdown '25: Is it Time to Ditch Spark Yet??
Mastering Spark: Session vs. DataFrameWriter vs. Table Configs
Should You Ditch Spark for DuckDb or Polars?
Breaking the Myth: Spark Isn’t as Scary as You'd Think (And Yes, It Supports SQL!)
Mastering Spark: Parallelizing API Calls and Other Non-Distributed Tasks
Mastering Spark: RDDs vs. DataFrames
Mastering Spark: Enhancing Job Visibility
From Databricks to Fabric: A Deep Dive into Spark Cluster Differences
Optimizing Spark: A Deep Dive into Optimized Write in Microsoft Fabric
Fabric Announcements at Build '24
The Fabric Concurrency Showdown: RunMultiple vs. ThreadPools
Confessions of a Spark Convert
Beyond Information Schema: Metadata Mastery in a Fabric Lakehouse
Cluster Configuration Secrets for Spark: Unlocking Parallel Processing Power
Querying Databases in Apache Spark: Pandas vs. Spark API vs. Pandas-on-Spark
Architecture
Automation
Automating V-Order: A Targeted Approach for Direct Lake Models
Install Data Software Like a Pro
Azure Synapse / SQL User Provisioning the Right Way
Benchmarking
Mastering Spark: Should You Infer Schema in Production?
Announcing: 🌊 LakeBench
The Small Data Showdown '25: Is it Time to Ditch Spark Yet??
Mastering Spark: The Art and Science of Table Compaction
Should You Ditch Spark for DuckDb or Polars?
Unlock Faster Writes in Delta Lake with Deletion Vectors
Mastering Spark: Parallelizing API Calls and Other Non-Distributed Tasks
To V-Order or Not: Making the Case for Selective Use of V-Order in Fabric Spark
Optimizing Spark: A Deep Dive into Optimized Write in Microsoft Fabric
The Fabric Concurrency Showdown: RunMultiple vs. ThreadPools
Querying Databases in Apache Spark: Pandas vs. Spark API vs. Pandas-on-Spark
Synapse Optimization Series: Table Distributions
Compaction
Concurrency
Why you might lease a blob: an intro to job locks
Mastering Spark: Parallelizing API Calls and Other Non-Distributed Tasks
The Fabric Concurrency Showdown: RunMultiple vs. ThreadPools
Cluster Configuration Secrets for Spark: Unlocking Parallel Processing Power
Configuration
Creating your first Spark Job Definition
Mastering Spark: Session vs. DataFrameWriter vs. Table Configs
From Databricks to Fabric: A Deep Dive into Spark Cluster Differences
Linked Service Property Magic
Cost Optimization
From Databricks to Fabric: A Deep Dive into Spark Cluster Differences
Confessions of a Spark Convert
Daft
Data Governance
Data Layout
How Incremental Liquid Clustering Works
Automating V-Order: A Targeted Approach for Direct Lake Models
To V-Order or Not: Making the Case for Selective Use of V-Order in Fabric Spark
Optimizing Spark: A Deep Dive into Optimized Write in Microsoft Fabric
Synapse Optimization Series: Table Distributions
Deletion Vectors
Delta Lake
How Incremental Liquid Clustering Works
Mastering Spark: DataFrameWriterV2 vs. DataFrameWriterV1
Mastering Spark: The Art and Science of Table Compaction
Mastering Spark: Session vs. DataFrameWriter vs. Table Configs
Should You Ditch Spark for DuckDb or Polars?
Unlock Faster Writes in Delta Lake with Deletion Vectors
Optimizing Spark: A Deep Dive into Optimized Write in Microsoft Fabric
Fabric Announcements at Build '24
Decoding Delta Lake Compatibility Between Fabric and Databricks
Delta Table Maintenance 101
Developer Tooling
Announcing: 🌊 LakeBench
Announcing the Microsoft Fabric Shape Library for Excalidraw
Elevate Your Code: Creating Python Libraries Using Microsoft Fabric (Part 1 of 2: Development)
Install Data Software Like a Pro
Diagramming
Distributed Locking
DuckDB
The Small Data Showdown '25: Is it Time to Ditch Spark Yet??
Should You Ditch Spark for DuckDb or Polars?
Interoperability
Yet Another Way to Connect to the SQL Endpoint / Warehouse via Python
Decoding Delta Lake Compatibility Between Fabric and Databricks
LLMs
Liquid Clustering
How Incremental Liquid Clustering Works
Mastering Spark: DataFrameWriterV2 vs. DataFrameWriterV1
Delta Table Maintenance 101
Metadata
Automating V-Order: A Targeted Approach for Direct Lake Models
Beyond Information Schema: Metadata Mastery in a Fabric Lakehouse
Migration
From Databricks to Fabric: A Deep Dive into Spark Cluster Differences
Generative AI and Spark: Leveraging LLMs for Accelerated Migrations
The SQL Decoder Ring for Replatforming to Fabric and Databricks
Decoding Delta Lake Compatibility Between Fabric and Databricks
Enhancing Your Lakehouse with Unity Catalog
Observability
Orchestration
Why you might lease a blob: an intro to job locks
Creating your first Spark Job Definition
Fabric Announcements at Build '24
The Fabric Concurrency Showdown: RunMultiple vs. ThreadPools
Cluster Configuration Secrets for Spark: Unlocking Parallel Processing Power
Linked Service Property Magic
Page
Pandas
Yet Another Way to Connect to the SQL Endpoint / Warehouse via Python
Querying Databases in Apache Spark: Pandas vs. Spark API vs. Pandas-on-Spark
Polars
The Small Data Showdown '25: Is it Time to Ditch Spark Yet??
Should You Ditch Spark for DuckDb or Polars?
Production Engineering
Notebooks, Spark Jobs, and the Hidden Cost of Convenience
Creating your first Spark Job Definition
Elevate Your Code: Creating Python Libraries Using Microsoft Fabric (Part 2 of 2: Packaging, Distribution, and Consumption)
Elevate Your Code: Creating Python Libraries Using Microsoft Fabric (Part 1 of 2: Development)
Python
Why you might lease a blob: an intro to job locks
Notebooks, Spark Jobs, and the Hidden Cost of Convenience
Creating your first Spark Job Definition
Announcing: 🌊 LakeBench
Elevate Your Code: Creating Python Libraries Using Microsoft Fabric (Part 2 of 2: Packaging, Distribution, and Consumption)
Mastering Spark: Creating Resiliency with Retry Logic
Mastering Spark: Parallelizing API Calls and Other Non-Distributed Tasks
Mastering Spark: RDDs vs. DataFrames
Yet Another Way to Connect to the SQL Endpoint / Warehouse via Python
Elevate Your Code: Creating Python Libraries Using Microsoft Fabric (Part 1 of 2: Development)
Generative AI and Spark: Leveraging LLMs for Accelerated Migrations
The SQL Decoder Ring for Replatforming to Fabric and Databricks
Cluster Configuration Secrets for Spark: Unlocking Parallel Processing Power
Python Packaging
Elevate Your Code: Creating Python Libraries Using Microsoft Fabric (Part 2 of 2: Packaging, Distribution, and Consumption)
Elevate Your Code: Creating Python Libraries Using Microsoft Fabric (Part 1 of 2: Development)
Query Optimization
Breaking the Myth: Spark Isn’t as Scary as You'd Think (And Yes, It Supports SQL!)
To V-Order or Not: Making the Case for Selective Use of V-Order in Fabric Spark
Fabric Announcements at Build '24
Data Toboggan - Winter Edition 2023
Synapse Optimization Series: Table Distributions
Resilience
Why you might lease a blob: an intro to job locks
Notebooks, Spark Jobs, and the Hidden Cost of Convenience
Mastering Spark: Creating Resiliency with Retry Logic
Row-Level Security
SQL
Breaking the Myth: Spark Isn’t as Scary as You'd Think (And Yes, It Supports SQL!)
Yet Another Way to Connect to the SQL Endpoint / Warehouse via Python
The SQL Decoder Ring for Replatforming to Fabric and Databricks
Beyond Information Schema: Metadata Mastery in a Fabric Lakehouse
RLS in Databricks Unity Catalog and Power BI
Querying Databases in Apache Spark: Pandas vs. Spark API vs. Pandas-on-Spark
Azure Synapse / SQL User Provisioning the Right Way
Data Toboggan - Winter Edition 2023
Synapse Optimization Series: Table Distributions
Schema Management
Mastering Spark: DataFrameWriterV2 vs. DataFrameWriterV1
Mastering Spark: Should You Infer Schema in Production?
Table Maintenance
How Incremental Liquid Clustering Works
Mastering Spark: The Art and Science of Table Compaction
Unlock Faster Writes in Delta Lake with Deletion Vectors
Delta Table Maintenance 101
Fabric (former tag)
Platform tags are now content tracks. Browse articles, then narrow by technology or topic.
Microsoft Fabric (former tag)
Platform tags are now content tracks. Browse articles, then narrow by technology or topic.
Databricks (former tag)
Platform tags are now content tracks. Browse articles, then narrow by technology or topic.
Azure Databricks (former tag)
Platform tags are now content tracks. Browse articles, then narrow by technology or topic.
Azure Synapse (former tag)
Platform tags are now content tracks. Browse articles, then narrow by technology or topic.
Azure Data Factory (former tag)
Platform tags are now content tracks. Browse articles, then narrow by technology or topic.
Dedicated SQL Pools (former tag)
Platform tags are now content tracks. Browse articles, then narrow by technology or topic.
Power BI (former tag)
Platform tags are now content tracks. Browse articles, then narrow by technology or topic.
Lakehouse (former tag)
This broad tag has been retired. Browse the article archive using specific technology and topic tags.