About the job Supervisor, Big Data Modeler
Purpose of the job
As a data engineer youll work as s a Big Data Modeler, you will play a vital
role enabling self-service business users, data analysts and data scientists
to extract the most value out of company data with the least effort by
delivering rich, optimized, scalable data models.
Duties and responsibilities
* Identifying data ingestion patterns and build framework to efficiently ingest the data to our Data Lake
* Performance tuning ingestion jobs to improve throughput
* Research opportunities for data acquisition and new uses for existing data
* Develop data set processes for data modeling, mining and production
* Integrate new data management technologies and software engineering tools into existing framework
* Collaborate with data engineers, architects and IT team members on project goals
Job Specifications:
Education
* Bachelor degree of Engineering in Computer Systems, or Computer Science.
Experience
* 3 to 5 years data engineering experience building data pipelines and systems.
* Experience in working with Hadoop technologies such as Spark
* Experience in working with data flow tools such as Nifi, Airflow,
* Prior experience with relational databases such as MYSQL
* Knowledge and understanding of SDLC and Agile/Scrum procedures, CI/CD and Automation is required
* Ability to use containers like dockers or kubernettes is a plus
* Ability to write SQL queries and use tools such as Hadoop, Tableau, QlikView, and other data reporting tools. Experience in transactional and data warehouse environments using MySQL, Hive, or other database systems. Must deeply understand joins, subqueries, window functions, etc.
Skills and abilities
* Strong ability to drive complex technical solutions deployed at an enterprise level; ability to drive big data technology adoption and changes through education and partnership with stakeholders
* Demonstrated experience in working with the vendor(s) and user communities to research and test new technologies to enhance the technical capabilities of existing Hadoop cluster
* Ability to negotiate, resolve, and prioritize complex issues and provide explanations and information to others on difficult issues, assess alternatives and implement long-term solutions
* Self-starter who can work with minimal guidance
* Strong communication skills
* Very good English both written & spoken.