Big Data Engineer → Senior Data Engineer

A realistic transition from Big Data Engineer to Senior Data Engineer: transferable skills, the gaps to close, a step-by-step learning plan and alternative routes. Estimated skill overlap 74%. Difficulty: Easy.

Difficulty: Easy6 stepsDuration: 4 to 8 monthsOverlap: 74%

Transition from Big Data Engineer to Senior Data Engineer.

Transferable skills

  • Data platform architecture
  • Python / Scala
  • Spark / distributed processing
  • Cloud data platforms
  • Streaming (Kafka)
  • Data modelling

Skills to learn

  • Orchestration (Airflow)
  • Mentoring

Step-by-step learning plan

  1. 1. Build on your Big Data Engineer experience

    2–4 weeks

    Leverage

    • Data platform architecture
    • Python / Scala
    • Spark / distributed processing
    • Cloud data platforms

    Map transferable skills — Data platform architecture, Python / Scala, Spark / distributed processing — against what a Senior Data Engineer needs.

  2. 2. Learn the core of the Senior Data Engineer role

    6–10 weeks

    Learn

    • Orchestration (Airflow)

    Technologies

    dbtBigQueryRedshift

    Cover the biggest gaps first: Orchestration (Airflow).

  3. 3. Master advanced Senior Data Engineer skills

    6–10 weeks

    Learn

    • Mentoring

    Technologies

    AWSGCPPrefect

    Handle the advanced topics: Mentoring.

  4. 4. Get productive with the Senior Data Engineer toolchain

    4–8 weeks

    Technologies

    dbtBigQueryRedshiftAWSGCPPrefect DagsterLinux

    Be hands-on with dbt, BigQuery, Redshift and the rest of the target stack.

  5. 5. Practise Senior Data Engineer work in real conditions

    4–8 weeks

    Ship a portfolio project or take on Senior Data Engineer tasks at work.

  6. 6. Get certified and position yourself as a Senior Data Engineer

    4–8 weeks

    Technologies

    Google Professional Data EngineerAWS Data Analytics SpecialtyDatabricks Certified Data Engineer

    Update your CV and LinkedIn around Senior Data Engineer, then apply to target roles.

Alternative routes