Horizontal is proud to be an Equal Opportunity and Affirmative Action Employer. We seek to provide employment opportunities to talented, qualified candidates regardless of race, color, sex/gender including gender identity and/or expression, national origin, religion, sexual orientation, disability, marital status, citizen status, veteran status, or any other protected classification under federal, state or local law.
In addition, Horizontal will provide reasonable accommodations for qualified individuals with disabilities. If you need to request a reasonable accommodation in order to complete the application or interview process, please contact us.
All applicants applying must be legally authorized to work in the country of employment.
Responsibilities
Overview:
- Architect and refine our Databricks-centric data platform, emphasizing scalable integrations and advanced entity resolution strategies.
- Lead the charge towards a data mesh infrastructure, promoting domain-centric design and decentralized data management.
- Enhance our DataOps capabilities, emphasizing data observability, discovery, and lineage to maintain data integrity and accuracy.
- Pioneer the adoption of Databricks data lakehouse architectures, focused on simplifying data management and enhancing data processing capabilities.
- Serve as a technical liaison among analytics engineers, BI developers, and analysts within the Databricks environment, integrating solutions like Segment.io.
- Develop and optimize SQL and Python/Scala scripts within Databricks for complex data transformation and integration tasks.
- Work closely with cross-functional teams to align data strategies with operational needs and objectives, incorporating advanced data resolution methodologies.
- Stay at the forefront of industry trends and technological advancements to ensure our Databricks platform remains innovative and effective.
Qualifications
Must Have:
- Bachelor’s
- Strong leadership skills and the ability to advocate for best practices in data management and processing.
- 5+ years of experience in a data engineering role, with history of implementing data platform modernization
- 2+ years of hands on databricks experience.
- In-depth understanding of Databricks, along with experience in other cloud data warehouses like Snowflake, Redshift, Big Query.
- Building data integration and orchestration within the databricks environment, with hands-on experience in technologies such as Delta Live Tables, CDC, dbt, airflow, Segment.io.
- Advanced skills in SQL, Python, and Scala, with an emphasis on their application within Databricks for complex data tasks.
- Knowledge of best practices in version control and CI/CD with Git and GitHub Actions; Agile project management with Jira and Confluence.
- Proficiency with the Azure data suite (Azure SQL, Data Factory, Synapse Analytics, Power BI).
- Solid understanding of data observability, discovery, and lineage, and their application in maintaining high data quality standards.
- Additional experience with AI and ML capabilities in the context of Databricks is a plus.
Horizontal Talent
US
