Robust pipelines start before the tool.

• 11 Jun
  Prompt caching: the one-line change that cuts 90% of LLM cost in production cache_control ephemeral, a 5-minute TTL, and how one line dropped 75% of the cost of my nightly pipeline.
  llm
• 10 Jun
  SQL is still the most important language in data engineering in 2026 Why the dev who outsources SQL to an ORM or pandas becomes the spreadsheet filler of the AI era.
  sql
• 04 Jun
  YouTube rate-limits its caption endpoint. Audio stays free. Why --sleep-subtitles 60 no longer fixes YouTube's HTTP 429 on captions, which endpoint still works, and what self-hosting with faster-whisper delivers vs paid SaaS.
  youtube
• 02 Jun
  Code Review of My Own Old Repo. Five Things I'd Change Today. I opened a two-year-old repo of mine. Found five anti-patterns in my own code. Honest review before continuing to cite it on my portfolio.
  code-review
• 30 May
  SLA, not trend: when batch, when streaming, when both The choice between batch and streaming is not cultural. It is technical: consumer SLA, cost, complexity. And almost nobody measures it before deciding.
  batch
• 30 May
  Data Flows Ep01: the concept that comes before any tool Knight Capital lost $440 million in 45 minutes because one server ran old code. Before any tool, there is a concept that separates a robust pipeline from a fragile one: data flow.
  zero-to-expert
• 24 May
  Airflow for 2 years: what I would do differently Two years of Airflow in production: finance, factory processes, marketing, analytics. List of what I would change if starting over, not doc theory.
  airflow
• 23 May
  Twenty AI concepts you need to understand in 2026 The complete visual guide to how AI works, from the basics to the frontier. Six sections, twenty concepts, no spare jargon.
  ai
• 17 May
  When the model should say 'I don't know' In 1998 LTCM lost $4.6 billion because its models trusted territory they had never seen. Twenty-seven years later my HMM was doing the same, at smaller scale. Here's what I learned to put between it and the trade.
  crypto
• 13 May
  Instrumenting lineage from scratch with Unity Catalog Unity Catalog captures lineage automatically, but the limits matter: UPDATE/DELETE/MERGE, RDDs, JDBC, multi-hop. What it covers and where most people get it wrong.
  databricks
• 12 May
  When Medallion Architecture gets in the way more than it helps Medallion became the default answer for any pipeline. Where it works, where it gets in the way, and the anti-pattern nobody talks about.
  medallion
• 02 May
  LGPD and ML models: what to do with data that has already become model weights A data subject requests deletion. You delete the row from the database. The model in production still carries the contribution from that CPF. What LGPD requires and what's actually possible.
  lgpd
• 30 Apr
  Delta Lake or Parquet? You're asking the wrong question Delta Lake doesn't compete with Parquet. It's built on top of it. The real decision is different, and getting it wrong silently corrupts data.
  delta-lake
• 23 Apr
  Sharpe Ratio -1.14 is an Engineering Win, Not a Failure Data leakage destroyed a +0.66 Sharpe ratio into -1.14. What I learned: 7-layer architecture, 202 tests, 90x better capital preservation than buy-and-hold.
  crypto
• 18 Apr
  Real data engineering content in Portuguese is rare. I'm going to help change that. Why a senior data engineer with experience across major Brazilian banks, a global-scale fintech and a big-tech project in Silicon Valley decided to start writing, and what you'll find here.
  manifesto
• 16 Apr
  LGPD at the ingestion layer: 4 principles that change your architecture LGPD isn't a checklist at the end of the pipeline. It's a design constraint from the first byte that comes in. What that means in practice.
  lgpd

No posts yet. All