Director / Data Manager: Data Engineer
Century City, Cape Town, Western Cape, 7441, South Africa On-site
Salary not listedJobFig found this opening at its original source and checks that it remains available.
About the role
The OpportunitySenior Data Engineer Who We Are Are you passionate about working on challenging business problems that can be solved by the intersection of people, data, technology, and AI? And do you want to be part of a fast- growing world-class team of AI experts who thrives under pressure and strives to go the extra mile to fuel the bold visions of our clients? Then we want to talk to you! Founded in South Africa and headquartered in the US, AdvanceGuidance is a premier AI consulting firm, trusted by our global client base to help them unlock the value in their data. Our cutting-edge AI solutions are designed to help our clients increase revenue, optimize operations, improve customer experiences, and gain a competitive edge in today’s fast-paced digital landscape. We deliver a comprehensive suite of AI solutions and services, including Natural Language Processing, GenAI, LLMs, Predictive Analytics, and Machine Learning, among others. Our dedication to adding value to our clients is a state of mind. This means we hire to an exceptionally high talent bar. Intellect isn’t a differentiator. Being a self-starter, willing to get stuck in, and deliver the best possible work, is. We only hire people with bags of ambition, energy and passion that comes with wanting to help build and grow a business, not just work for one. What You’ll Be Doing • Building modern database structures storing terabytes of data used to develop and maintain machine learning models in a cloud environment. • Collaborate with Software Engineering stakeholders to define and implement new data powered solutions, and deploy and maintain them in production. • Supporting best practices in cataloguing, characterising, classifying and communicating accessible data and initiatives relative to data creation so teams can easily access the data. • Creating and owning best practices in terms of data ‘organisation’, standards and versioning including in a Cloud environment. • Own data privacy and security standards according to GDPR and applying these to production systems. • Being a driving force behind data creation and storage initiatives for Cloud applications across the organisation. • Embracing modern DevOps principles to development and working in a setting where code is expected to be shared and peer-reviewed. • Evangelise software engineering best practices and proactively identify technical debt and other issues to improve our products. • Seeking out ways to automate and accelerate development of data driven products, and then implementing them. • Learning new skills and technologies through courses, reading industry papers, or pair-programming with co-workers on other teams, to ensure the team is always cutting edge. What we’re looking for • 5 yrs + experience w/ data wrangling/data engineering, implementing and supporting robust data transformation pipelines in a production Cloud environment. • Experience with RDBMS systems for OLTP workloads, such as Postgres or MySQL • Experience architecting data (optimise/restructure data for fast analysis). • Experience implementing methods to ensure large-scale ( big-data ) quality and security. • Experience with software design patterns, test-driven development. • Experience working with big data technologies such as Spark, Hadoop, Presto, Hive, AWS. Athena and Terraform. • Experience with dynamic languages, such as Python or R. • Experience with container technology, like Docker. • Experience of JavaScript frameworks like Vue, Node.js. • Experience in helping develop/deploy technology products (major plus). • Experience working in a diverse team. • Nice to have: worked with time-series datasets, big data systems (terabytes or more) in an AWS environment. The Opportunity • Work closely with other data, machine learning and Cloud enthusiasts. • Enjoy your work and life harmony. • Leave your mark in a thriving industry. • Apply your talent to tackling new challenges everyday. • Be surprised at how much you will learn. • Open and inclusive company culture. Who you are • Passion and excitement about making a difference for clients using digital technologies. • A getting-things-done mindset with a desire to turn knowledge into solutions people will rely on. • A university degree in related field.
What you'll bring
- 5 yrs + experience w/ data wrangling/data engineering, implementing and supporting robust data transformation pipelines in a production Cloud environment.
- Experience with RDBMS systems for OLTP workloads, such as Postgres or MySQL
- Experience architecting data (optimise/restructure data for fast analysis).
- Experience implementing methods to ensure large-scale ( big-data ) quality and security.
- Experience with software design patterns, test-driven development.
- Experience working with big data technologies such as Spark, Hadoop, Presto, Hive, AWS.
- Athena and Terraform.
- Experience with dynamic languages, such as Python or R.
- Experience with container technology, like Docker.
- Experience of JavaScript frameworks like Vue, Node.js.
- Experience working in a diverse team.
- Passion and excitement about making a difference for clients using digital technologies.