Azure Junior Data Engineer - Travel Experience Network - Queensland

Travel Experience Network | Posted 12-08-2019

Queensland (Web Programmer)


Travel Experience Network-QueenslandJob no: 504179
Brand: Travel Experience Network
Work type: Full time
Location: Queensland
Categories: Business Support

Job Purpose

The Travel Group (TTG), within the Flight Centre Travel Group (FCTG), is on the hunt for our next Junior Azure Data Engineer gun. This is an exciting opportunity and for you to bring your energy and unique perspective to TTG, you will have the opportunity to grow with us professionally and personally and you will be part of a team that genuinely cares about helping you succeed. Not to mention, you will be working with data that will impact our customers travel experience.

You will be a part of the TTG business within the TTG IT Division. As a Data Engineer, you will use and be exposed to the latest Microsoft Tech! You will help collect, store and aid our data warehousing and data modelling crew to support business outcomes. You will be a critical part of the data supply chain, ensuring business partners have access to right analytics to help drive their business. You will also help design and build data pipelines utilising data factory and databricks notebooks to transform millions of records into action data models that enable data insights.

To be successful, the candidate is expected to have:

  • A passion for Azure Microsoft stack and willingness to grow
  • Knowledge of Azure Data Factory, Azure Databricks, Azure Data Lake, Azure SQL DW / SQLDB, Azure SQL and T-SQL
  • Ability to conduct data profiling, cataloging, data lineage and mapping for technical design and construction of technical data flows
  • Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code fundamentals
  • Knowledge of Master Data Management (MDM) and Data Quality tools and processes
  • Working experience with VS Code and GIT

The Team

The TTG BI Data warehousing Team provides a range of services to TTG business partners and other internal stakeholders to FCTG. The team is responsible for the TTG Data warehouse development, DevOps (CI/CD) pipelines, ETL development and are gate keepers of data within the TTG network. We ensure that our products are designed to meet best practice that exceeds internal customer expectations. To help achieve this we operate in an Agile environment and we pride ourselves using the latest tech be it in DevOps or Data Engineering.

Accountabilities

ETL / Data Ingestion (Data Lake, Blob, Data Bricks/Factory)

  • Development of data pipelines and re-usable notebooks use of Data factory and Data bricks
  • Coding in SQL/Python, leveraging the Spark processing engine.
  • Ensure design and structure of the data lake as per policy.
  • Develop and maintain data integration solutions to load and transform data.
  • Ensure standards to create reusable frameworks for data load and error handling is used.
  • Ensure incremental data loading patterns are applied.
  • Extracting data from various sources (API, SFTP, Azure BLOB, S3 bucket)
  • Assist in standard/governance and writing of technical documentation

Data Warehouse

  • Work closely with internal BI team and domain knowledge experts to help scope upcoming technical work.
  • Assist technical scoping document on data sourcing, data structure and data model (conceptual, logical, and physical).
  • Learn to implement and maintain data warehouse (Kimball) model and data marts
  • Knowledge on performance tuning on the data warehouse.
  • Ensuring Data Lineage is updated and assisting with MDM
  • Assisting with security, data protection and ensuring FCTG policy is maintained

DevOps and Governance

  • Ensuring you operate within a structured environment (Prod, Stage and Dev) and meet coding guidelines/standards
  • Working as an apart of agile development team with sprint planning and cycles
  • Manage develop and maintaining code within Azure Devops
  • Help maintain code quality, organisation and automation
  • Develop and execute test plans and scenarios including data design and data extract/transform
  • Be involved and complete QA items and peer reviews for sign off.
  • Ensure you learn and develop yourself to keep abreast of industry best practice

Skills and Experience

Essential

  • Familiar with Agile/Scrum Development Life Cycle process including analysis and design, software development, testing, implementation, maintenance, and support
  • Strong in SQL, Databases, and data modelling methodologies (Kimball)
  • Knowledge of how changed data capture (CDC) works and how to make use of this within DWH
  • Knowledge of Azure Data Factory, Azure Databricks, Azure Data Lake, Azure SQL DW / SQLDB, Azure SQL and T-SQL
  • Ability to conduct data profiling, cataloging, data lineage and mapping for technical design and construction of technical data flows
  • Knowledge of Master Data Management (MDM) and Data Quality tools and processes
  • Knowledge of Dev-Ops processes (including CI/CD) and Infrastructure as code fundamentals
  • Version Control – working knowledge of Azure DevOps process, practical use of tools such as Azure GIT
  • Working experience with Visual Studio and basic PowerShell Scripting
  • Documentation – well vised in creating technical design documents, data flow diagrams
  • Recommended limitations in current technical development process and advised improvisations by eliminating redundant processes
  • Drafted technical specification documentations by mapping Business requirements
  • Ensured all tasks met desired quality standards by using standardized development practices
  • Develop & support Relational Database (T-SQL) and show recent querying experience
  • Degree/relevant qualification
  • Excellent verbal and written communication skill with a focus on customer service

Desirable

  • Workflow (JIRA, Azure Boards) & Documentation (Confluence)
  • C#, Phyton, USQL, Powershell and .NET
  • Min 1-2 years’ experience in a commercial environment

Benefits include:

  • Generous remuneration structure
  • Travel discounts, in-house financial and health services, access to internal 24/7 gym
  • Global career opportunities in a network of brands and businesses
  • Ongoing training and professional development
  • Fun and flexible work environment
  • Proud Corporate Social Responsibility platform through the Flight Centre Foundation, Responsible Travel Charter, and Brighter Futures programs
  • Monthly awards nights, conferences, industry/social events, and the opportunity to attend global balls