Lesson 03 of 7 · The layers move upstream
Same layers. Earlier.
Your bronze, silver and gold aren't going anywhere. They're just moving house, upstream.
What happens to the Power Query layers?
Today the kit's report file holds bronze, silver and gold queries in Power Query. Only gold loads into the model. In Fabric the model will read Delta tables from the lake.
So where do the bronze, silver and gold steps go?
Same layers, one level earlier
🖥️ Today, in Power Query
Bronze, silver and gold are queries. Bronze and silver don't load. Gold loads into the model on every refresh, and each refresh runs every step again.
🌊 In Fabric, in the lake
Bronze is the file in Files. Silver and gold are Delta tables written by a or a , run by a . The model just reads gold.
The bonus: gold now exists once, as tables anyone can query. Another model, a notebook or a SQL check can use the same gold without repeating the transformations.
Which tool for which job?
Fabric gives you three main tools for the layers. Most projects use all three.
You know M, the data is small, and you want to reuse the kit's existing queries
The data is large and the logic is complex; you want the code reviewed line by line in Git
Every night: run the dataflow, then the notebook, and stop and alert if either fails
A quick fix to trim spaces in the commodity names
Move the kit's layers
Tap the steps in order.
Start with the first step.
The calculated column
The kit's model has a calculated column, Criticality band, on the Commodity table. What happens to it?
Three things to keep
- 1The bronze, silver and gold layers move upstream and become real tables in the lake.
- 2 for M you already have, for big or complex logic, to run them in order.
- 3Anything the model used to compute per row, like a calculated column, moves upstream too.