JR-213103 Data Engineer Lead

Hello, our name is Customertimes, but you can call us CT. We are a global digital engineering, product development, and technology consulting company. Headquartered in New York, we have a team of 1000+ experts working across 65+ countries.

About this job:

We are seeking a hands-on platform engineer to architect and build the self-service capabilities our development teams need for their work on Databricks. The goal is that those teams provision their own data, infrastructure, and applications through governed, automated paths instead of filing tickets with the platform team. Platform engineering is a new discipline here, so the role spans defining the approach, building it, and training our team to carry it forward after the engagement ends.

Location: Brazil, Argentina, Colombia, Peru, Uruguay, Chile, Canada (remote)

How you'll make an impact:

• Lead the build of the developer self-service capabilities — bundle templates, provisioning scripts, and onboarding documentation that let domain teams stand up catalogs, schemas, service principals, and Lakebase databases themselves, replacing the current pattern of individually filed, often duplicated manual infrastructure tickets;

• Lead governance-as-code end to end — designing policy, building the validator/resolver engine, and extending enforcement to provisioning paths not currently covered (e.g., catalog-level grants declared in Terraform, which today bypass the same anti-escalation controls applied everywhere else);

• CI/CD deployment platform standardization — own the CI/CD deployment platform: bring reusable workflows and composite actions in-house, complete the move to OIDC federation so deploys carry no stored credentials, and hold all domain repos on a single policy version;

• Design least-privilege automation — scoped bot/service credentials with explicit, tested boundaries rather than broad standing access;

• Upskill the team currently covering this work — provide hands-on mentorship and pair with staff who are performing manual, UI-driven infrastructure administration today, moving them toward Infrastructure as Code and self-service patterns through direct collaboration, documentation, and internal training;

• Evaluate and pilot new platform capabilities — assess newer Databricks platform features (e.g., native secrets management) in a controlled pilot before broader adoption;

• Maintain platform documentation and migration tooling — getting-started guides and retrofit/migration scripts for legacy repositories so onboarding a new domain does not require hand-holding from the platform team.

What makes you a great fit:

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering, or a related field;

• 6+ years of experience in Data Engineering or Big Data roles;

• 3+ years of hands-on experience with Databricks;

• Experience in the insurance industry;

• Strong knowledge of the AWS cloud platform;

• Exposure to Large Language Models (LLMs), Generative AI concepts, prompt engineering, and AI-assisted development tools.

Core platform skills:

• Databricks Asset Bundles (DABs) authoring — designing reusable, parameterized bundle templates that let domain teams scaffold new projects rather than hand-building them;

• Governance-as-code — machine-readable policy design (classification-driven grant matrices, role/binding resolution), paired with a static pre-deploy validator and deploy-time enforcement, so access rules are enforced automatically rather than reviewed manually;

• Unity Catalog access model design — least-privilege, tiered access (admin/develop/read) with environment- and classification-aware enforcement, including closing gaps where infrastructure-as-code paths (e.g., Terraform-declared catalog grants) fall outside existing governance coverage;

• Infrastructure as Code — Terraform for platform-level resources (catalogs, workspace infrastructure) alongside Databricks Asset Bundles for domain-level resources;

• CI/CD platform engineering — GitHub Actions at scale, including building and owning reusable workflows and composite actions, implementing OIDC federation to replace stored credentials, and keeping many domain repositories on one shared workflow and policy version;

• OLTP/Lakebase provisioning — end-to-end lifecycle management (service principals, schema-level read/write roles, Data API enablement) exposed as a self-service workflow instead of one-off manual setup;

• Automation and tooling development — Python CLI tooling, validators, and test suites; least-privilege automation design (scoped bot credentials, smoke-tested boundaries) instead of standing broad access;

• Compute governance — cluster policies and Databricks Runtime version planning/rollout across dev/QA/prod, standardizing configurations instead of per-team ad hoc setup.

Other skills:

• English — excellent written and verbal communication skills;

• Ability to work in a global, multinational company;

• Ability to lead conversations with both technical and business representatives;

• Proven ability to work both independently and as part of an international project team.

If you're looking to contribute to global projects, grow your skills, shape the way you work, and be part of a connected international community — Customertimes is the place to build your next career chapter. Are you ready to spark the tech with us?

Job ID: JR-213103

Originally posted on Himalayas