Why dbt Wins for Transformation

Talend was built for a world where data lived in on-premise databases and transformation happened in transit — extract, transform, then load. That model made sense in 2010. It doesn't make sense when your warehouse is Snowflake or BigQuery and you're paying for compute that sits idle while Talend does the heavy lifting on a server somewhere.

dbt flips this. Data lands in the warehouse first via Fivetran or Airbyte — raw, untouched. Transformation happens inside the warehouse using SQL. The result: your compute scales with your data, your transformation logic lives in Git, every model is versioned and tested, and your analysts can read it without a Talend Studio license.

After migrating 12+ clients off Talend, the infrastructure cost reduction alone averages significant — across our broader client base we've saved $2.3M+ in infrastructure. The performance gains are consistent too: 67% average query speed improvement once workloads move into warehouse-native compute.

Before You Write a Single dbt Model: Do the Audit

The most common migration mistake is jumping straight to converting jobs. Don't.

Spend a week cataloguing everything in your Talend environment:

What to document for each job:

• Job name and purpose

• Source systems it reads from

• Transformations applied (tMap logic, filters, aggregations)

• Destination tables or files

• Schedule and dependencies

• Who owns it and when it was last touched

You'll find three categories: jobs that map cleanly to dbt SQL models, jobs that involve iteration or file processing that need a different solution, and jobs that nobody is sure why they exist anymore. That third category is usually 20–30% of the total. Don't migrate those — retire them.

The Migration: Four Phases

Phase 1 — Extraction Stays Out of dbt

dbt doesn't extract data. That's not what it's for. If Talend was also handling your extraction — pulling from APIs, databases, SFTP — you need to replace that separately before touching dbt.

We use Fivetran for managed connectors (Salesforce, Shopify, HubSpot, Google Ads) and Airbyte for custom or self-hosted sources. Both land raw data into Snowflake or BigQuery without transformation. Raw layer stays raw.

# sources.yml — declare your raw tables as dbt sources
version: 2

sources:
  - name: raw_shopify
    database: your_warehouse
    schema: raw
    tables:
      - name: orders
      - name: customers
      - name

# sources.yml — declare your raw tables as dbt sources
version: 2

sources:
  - name: raw_shopify
    database: your_warehouse
    schema: raw
    tables:
      - name: orders
      - name: customers
      - name

# sources.yml — declare your raw tables as dbt sources
version: 2

sources:
  - name: raw_shopify
    database: your_warehouse
    schema: raw
    tables:
      - name: orders
      - name: customers
      - name

This replaces every tDBInput component in Talend. The data is already in the warehouse. dbt just needs to know where.

Phase 2 — Convert tMap Logic to SQL Models

The bulk of migration work is here. Every tMap component in Talend becomes a dbt model: a .sql file that selects, joins, filters, and aggregates.

Talend tMap equivalent in dbt:

-- models/staging/stg_orders.sql
with source as (
    select * from {{ source('raw_shopify', 'orders') }}
),

renamed as (
    select
        id                      as order_id,
        customer_id,
        created_at              as order_date,
        total_price             as revenue_usd,
        financial_status        as payment_status,
        fulfillment_status
    from source
    where financial_status != 'voided'
)

select * from

-- models/staging/stg_orders.sql
with source as (
    select * from {{ source('raw_shopify', 'orders') }}
),

renamed as (
    select
        id                      as order_id,
        customer_id,
        created_at              as order_date,
        total_price             as revenue_usd,
        financial_status        as payment_status,
        fulfillment_status
    from source
    where financial_status != 'voided'
)

select * from

-- models/staging/stg_orders.sql
with source as (
    select * from {{ source('raw_shopify', 'orders') }}
),

renamed as (
    select
        id                      as order_id,
        customer_id,
        created_at              as order_date,
        total_price             as revenue_usd,
        financial_status        as payment_status,
        fulfillment_status
    from source
    where financial_status != 'voided'
)

select * from

Every staging model does one thing: rename columns, cast types, filter obvious garbage. No joins. No aggregations. One source, one model.

Joins and business logic go in the mart layer:

-- models/marts/mart_revenue_daily.sql
with orders as (
    select * from {{ ref('stg_orders') }}
),

customers as (
    select * from {{ ref('stg_customers') }}
),

final as (
    select
        o.order_date,
        c.country,
        c.customer_segment,
        count(distinct o.order_id)  as order_count,
        sum(o.revenue_usd)          as total_revenue
    from orders o
    left join customers c using (customer_id)
    group by 1, 2, 3
)

select * from

-- models/marts/mart_revenue_daily.sql
with orders as (
    select * from {{ ref('stg_orders') }}
),

customers as (
    select * from {{ ref('stg_customers') }}
),

final as (
    select
        o.order_date,
        c.country,
        c.customer_segment,
        count(distinct o.order_id)  as order_count,
        sum(o.revenue_usd)          as total_revenue
    from orders o
    left join customers c using (customer_id)
    group by 1, 2, 3
)

select * from

-- models/marts/mart_revenue_daily.sql
with orders as (
    select * from {{ ref('stg_orders') }}
),

customers as (
    select * from {{ ref('stg_customers') }}
),

final as (
    select
        o.order_date,
        c.country,
        c.customer_segment,
        count(distinct o.order_id)  as order_count,
        sum(o.revenue_usd)          as total_revenue
    from orders o
    left join customers c using (customer_id)
    group by 1, 2, 3
)

select * from

Phase 3 — Handle Iteration with Airflow, Not dbt

Talend's tFlowToIterate — looping row-by-row over a dataset — has no direct dbt equivalent, and that's by design. dbt is set-based. SQL processes sets. Iteration belongs in the orchestration layer.

If you have Talend jobs that loop over a list of clients, dates, or files and run different logic for each, move that logic to Apache Airflow:

• Airflow generates the dynamic parameters (client IDs, date ranges)

• dbt models accept those as variables: {{ var('client_id') }}

• Airflow triggers dbt runs with the right variables per iteration

This is a cleaner separation of concerns than anything Talend offered.

Phase 4 — Replace Data Quality Checks with dbt Tests

Talend's quality components (tDataQualityOutput, custom tMap filters) get replaced with dbt's native test framework. Add tests to every model:

# models/staging/stg_orders.yml
version: 2

models:
  - name: stg_orders
    columns:
      - name: order_id
        tests:
          - unique
          - not_null
      - name: payment_status
        tests:
          - accepted_values:
              values: ['paid', 'pending', 'refunded', 'partially_refunded']
      - name: customer_id
        tests:
          - not_null
          - relationships:
              to: ref('stg_customers')
              field

# models/staging/stg_orders.yml
version: 2

models:
  - name: stg_orders
    columns:
      - name: order_id
        tests:
          - unique
          - not_null
      - name: payment_status
        tests:
          - accepted_values:
              values: ['paid', 'pending', 'refunded', 'partially_refunded']
      - name: customer_id
        tests:
          - not_null
          - relationships:
              to: ref('stg_customers')
              field

# models/staging/stg_orders.yml
version: 2

models:
  - name: stg_orders
    columns:
      - name: order_id
        tests:
          - unique
          - not_null
      - name: payment_status
        tests:
          - accepted_values:
              values: ['paid', 'pending', 'refunded', 'partially_refunded']
      - name: customer_id
        tests:
          - not_null
          - relationships:
              to: ref('stg_customers')
              field

Run dbt test in your CI pipeline. If a test fails, the run stops. Bad data never reaches a dashboard. This alone eliminates the class of bugs that Talend environments silently propagate for months.

Run Both Systems in Parallel During Transition

Don't cut Talend off on day one. For each migrated job, run the dbt model and the Talend job in parallel for two to four weeks. Compare row counts, key metrics, and aggregate totals. When they match consistently, retire the Talend job.

This parallel validation phase is where most migrations succeed or fail. Teams that skip it spend months chasing discrepancies in production. Teams that run it systematically ship with confidence.

What the Stack Looks Like After Migration