Andrew Ridgway
GitHub
commit
f0cad218de
119
src/content/resume.md
Normal file
119
src/content/resume.md
Normal file
@ -0,0 +1,119 @@
|
||||
Title: A Resume
|
||||
Date: 2024-02-23 20:00
|
||||
Modified: 2024-03-13 20:00
|
||||
Category: Resume
|
||||
Tags: Cover Letter, Resume
|
||||
Slug: cover-letter
|
||||
Authors: Andrew Ridgway
|
||||
Summary: A Summary of My work Experience
|
||||
|
||||
# OVERVIEW
|
||||
I am a Senior Data Engineer looking to transition my skills to Data and Solution
|
||||
Architecting as well as project management. I have spent the better part of the
|
||||
last decade refining my abilities in taking business requirements and turning
|
||||
those into actionable data engineering, analytics, and software projects with
|
||||
trackable metrics. I believe in agnosticism when it comes to coding languages
|
||||
and have experimented in my own time with many different languages. In my
|
||||
career I have used Python, .NET, PowerShell, TSQL, VB and SAS (multiple
|
||||
products) in an Enterprise capacity. I also have experience using Google Cloud
|
||||
Platform and AWS tools for ETL and data platform development as well as git
|
||||
for version control and deployment using various IAC tools. I have also
|
||||
conducted data analysis and modelling on business metrics to find relationships
|
||||
between both staff and customer behavior and produced actionable
|
||||
recommendations based on the conclusions. In a private context I have also
|
||||
experimented with C, C# and Kotlin I am looking to further my career by taking
|
||||
my passion for data engineering and analysis as well as web and software
|
||||
development and applying it in a strategic context.
|
||||
|
||||
# SKILLS & ABILITIES
|
||||
- Python (scripting, compiling, notebooks – Sagemaker, Jupyter)
|
||||
- git
|
||||
- SAS (Base, EG, VA)
|
||||
- Various Google Cloud Tools (Data Fusion, Compute Engine, Cloud Functions)
|
||||
- Various Amazon Tools (EC2, RDS, Kinesis, Glue, Redshift, Lambda, ECS, ECR, EKS)
|
||||
- Streaming Technologies (Kafka, Hive, Spark Streaming)
|
||||
- Various DB platforms both on Prem and Serverless (MariaDB/MySql,
|
||||
- Postgres/Redshift, SQL Server, RDS/Aurora variants)
|
||||
- Various Microsoft Products (PowerBI, TSQL, Excel, VBA)
|
||||
- Linux Server Administration (cron, bash, systemD)
|
||||
- ETL/ELT Development
|
||||
- Basic Data Modelling (Kimball, SCD Type 2)
|
||||
- IAC (Cloud Formation, Terraform)
|
||||
- Datahub Deployment
|
||||
- Dagster Orchestration Deployments
|
||||
- DBT Modelling and Design Deployments
|
||||
- Containerised and Cloud Driven Data Architecture
|
||||
|
||||
# EXPERIENCE
|
||||
## Cloud Data Architect, Redeye Apps
|
||||
_May 2022 - Present_
|
||||
- Greenfields Research, Design and Deployment of S3 datalake (Parquet)
|
||||
- AWS DMS, S3, Athena, Glue
|
||||
- Research Design and Deployment of Catalog (Datahub)
|
||||
- Design of Data Governance Process (Datahub driven)
|
||||
- Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)
|
||||
- CI/CD design and deployment of modelling and orchestration using Gitlab
|
||||
- Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy
|
||||
- Design of ETL/Pipelines (DBT)
|
||||
- Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)
|
||||
## Data Engineer, TechConnect IT Solutions
|
||||
_August 2021 – May 2022_
|
||||
- Design of Cloud Data Batch ETL solutions using Python (Glue)
|
||||
- Design of Cloud Data Streaming ETL solution using Python (Kinesis)
|
||||
- Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs
|
||||
- Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)
|
||||
- Apply Prebuilt ML models to software solutions (Sagemaker)
|
||||
- Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)
|
||||
- API testing and development (gRPC, Rest)
|
||||
## Enterprise Data Warehouse Developer, Auto and General Insurance
|
||||
_August 2019 - August 2021_
|
||||
- ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL
|
||||
- Bringing new data to the business to analyse for new insights
|
||||
- Redeveloped Version Control and brought git to the data team
|
||||
- Introduced python for API enablement in the Enterprise Data Warehouse
|
||||
- Partnering with the business to focus data project on actual need and translating into technical requirements
|
||||
|
||||
## Business Analyst, Auto and General Insurance
|
||||
_January 2018 - August 2019_
|
||||
- Automate Service Performance Reporting using PowerShell/VBA/SAS
|
||||
- Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting
|
||||
- Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends
|
||||
- Technologies used include VBA, PowerShell, SQL, Web API’s, SAS
|
||||
- Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel
|
||||
- Gather Requirements to build meaningful reporting solutions
|
||||
- Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.
|
||||
|
||||
## Forecasting and Capacity Analyst, Auto and General Insurance
|
||||
_January 2017 – January 2018_
|
||||
- Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers
|
||||
- This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.
|
||||
- Forecast inbound workload requirements for the Auto and General sales call center using time series analysis
|
||||
- Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation
|
||||
- Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy
|
||||
- Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders
|
||||
- Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave
|
||||
|
||||
## Senior HR Performance Analyst, Queensland Department of Justice and Attorney General
|
||||
_June 2016 - January 2017_
|
||||
- Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics
|
||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives
|
||||
|
||||
## Workforce Business Analyst, Queensland Department of Justice and Attorney General
|
||||
_July 2015 – June 2016_
|
||||
- Develop and refine current workforce analysis techniques and databases
|
||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||
- Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit
|
||||
- Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required
|
||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives
|
||||
|
||||
# EDUCATION
|
||||
- 2011 Bachelor of Business Management, University of Queensland
|
||||
- 2008 Bachelor of Arts, University of Queensland
|
||||
|
||||
# REFERENCES
|
||||
- Anthony Stiller Lead Developer, Data warehousing, Queensland Health
|
||||
_0428 038 031_
|
||||
- Jaime Brian Head of Cloud Ninjas, TechConnect
|
||||
_0422 012 17_
|
||||
|
||||
@ -47,7 +47,7 @@
|
||||
<meta property="og:type" content="article">
|
||||
<meta property="article:author" content="">
|
||||
<meta property="og:url" content="http://localhost:8000/cover-letter.html">
|
||||
<meta property="og:title" content="A Cover Letter">
|
||||
<meta property="og:title" content="A Resume">
|
||||
<meta property="og:description" content="">
|
||||
<meta property="og:image" content="http://localhost:8000/">
|
||||
<meta property="article:published_time" content="2024-02-23 20:00:00+10:00">
|
||||
@ -86,12 +86,14 @@
|
||||
<div class="row">
|
||||
<div class="col-lg-8 col-lg-offset-2 col-md-10 col-md-offset-1">
|
||||
<div class="post-heading">
|
||||
<h1>A Cover Letter</h1>
|
||||
<h1>A Resume</h1>
|
||||
<span class="meta">Posted by
|
||||
<a href="http://localhost:8000/author/andrew-ridgway.html">Andrew Ridgway</a>
|
||||
on Fri 23 February 2024
|
||||
</span>
|
||||
|
||||
Translations:
|
||||
<a href="http://localhost:8000/cover-letter.html" hreflang="en">en</a>
|
||||
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
@ -104,19 +106,113 @@
|
||||
<div class="col-lg-8 col-lg-offset-2 col-md-10 col-md-offset-1">
|
||||
<!-- Post Content -->
|
||||
<article>
|
||||
<p>To whom it may concern</p>
|
||||
<p>My name is Andrew Ridgway and I am a Data and Technology professional looking to embark on the next step in my career.</p>
|
||||
<p>I have over 10 years’ experience in System and Data Architecture, Data Modelling and Orchestration, Business and Technical Analysis and System and Development Process Design. Most of this has been in developing Cloud architectures and workloads on AWS and GCP Including ML workloads using Sagemaker. </p>
|
||||
<p>In my current role I have Proposed, Designed and built the data platform currently used by business. This includes internal and external data products as well as the infrastructure and modelling to support these. This role has seen me liaise with stakeholders of all levels of the business from Analysts in the Customer Experience team right up to C suite executives and preparing material for board members. I understand the complexity of communicating complex system design to different level stakeholders and the complexities of involved in communicating to both technical and less technical employees particularly in relation to data and ML technologies. </p>
|
||||
<p>I have also worked as a technical consultant to many businesses and have assisted with the design and implementation of systems for a wide range of industries including financial services, mining and retail. I understand the complexities created by regulation in these environments and understand that this can sometimes necessitate the use of technologies and designs, including legacy systems and designs, I wouldn’t normally use. I also have a passion of designing systems that enable these organisations to realise the benefits of CI/CD on workloads they would not traditionally use this capability. In particular I took a very traditional legacy Data Warehousing team and implemented a solution that meant version control was no longer controlled by a daily copy and paste of folders with dates on major updates. My solution involved establishing guidelines of use of git version control so that this could happen automatically as people committed new code to the core code base. As I have moved into cloud architecture I have made sure to use best practice and ensure everything I build isn’t considered production ready until it is in IAC and deployed through a CI/CD pipeline.</p>
|
||||
<p>In a personal capacity I am an avid tech and ML enthusiast. I have designed my own cluster including monitoring and deployment that runs several services that my family uses including chat and DNS and am in the process of designing a “set and forget” system that will allows me to have multi user tenancies on hardware I operate that should enable us to have the niceties of cloud services like email, storage and scheduling with the safety of knowing where that data is stored and exactly how it is used. I also like to design small IoT devices out of Arduino boards allowing me to monitor and control different facets of our house like temperature and light. </p>
|
||||
<p>Currently I am working on a project to merge my skill in SQL Modelling and Orchestration with GPT API’s to try and lessen that burden. You can see some of this work in its very early stages here:
|
||||
(gpt-sql-generator)[https://github.com/armistace/gpt-sql-generator]
|
||||
(dbt_sources_generator)[https://github.com/armistace/datahub_dbt_sources_generator]</p>
|
||||
<p>I look forward to hearing from you soon.</p>
|
||||
<p>Sincerely,</p>
|
||||
<hr>
|
||||
<p>Andrew Ridgway</p>
|
||||
<h1>OVERVIEW</h1>
|
||||
<p>I am a Senior Data Engineer looking to transition my skills to Data and Solution
|
||||
Architecting as well as project management. I have spent the better part of the
|
||||
last decade refining my abilities in taking business requirements and turning
|
||||
those into actionable data engineering, analytics, and software projects with
|
||||
trackable metrics. I believe in agnosticism when it comes to coding languages
|
||||
and have experimented in my own time with many different languages. In my
|
||||
career I have used Python, .NET, PowerShell, TSQL, VB and SAS (multiple
|
||||
products) in an Enterprise capacity. I also have experience using Google Cloud
|
||||
Platform and AWS tools for ETL and data platform development as well as git
|
||||
for version control and deployment using various IAC tools. I have also
|
||||
conducted data analysis and modelling on business metrics to find relationships
|
||||
between both staff and customer behavior and produced actionable
|
||||
recommendations based on the conclusions. In a private context I have also
|
||||
experimented with C, C# and Kotlin I am looking to further my career by taking
|
||||
my passion for data engineering and analysis as well as web and software
|
||||
development and applying it in a strategic context.</p>
|
||||
<h1>SKILLS & ABILITIES</h1>
|
||||
<ul>
|
||||
<li>Python (scripting, compiling, notebooks – Sagemaker, Jupyter)</li>
|
||||
<li>git</li>
|
||||
<li>SAS (Base, EG, VA)</li>
|
||||
<li>Various Google Cloud Tools (Data Fusion, Compute Engine, Cloud Functions)</li>
|
||||
<li>Various Amazon Tools (EC2, RDS, Kinesis, Glue, Redshift, Lambda, ECS, ECR, EKS)</li>
|
||||
<li>Streaming Technologies (Kafka, Hive, Spark Streaming)</li>
|
||||
<li>Various DB platforms both on Prem and Serverless (MariaDB/MySql,</li>
|
||||
<li>Postgres/Redshift, SQL Server, RDS/Aurora variants)</li>
|
||||
<li>Various Microsoft Products (PowerBI, TSQL, Excel, VBA)</li>
|
||||
<li>Linux Server Administration (cron, bash, systemD)</li>
|
||||
<li>ETL/ELT Development</li>
|
||||
<li>Basic Data Modelling (Kimball, SCD Type 2)</li>
|
||||
<li>IAC (Cloud Formation, Terraform)</li>
|
||||
<li>Datahub Deployment</li>
|
||||
<li>Dagster Orchestration Deployments</li>
|
||||
<li>DBT Modelling and Design Deployments</li>
|
||||
<li>Containerised and Cloud Driven Data Architecture</li>
|
||||
</ul>
|
||||
<h1>EXPERIENCE</h1>
|
||||
<h2>Cloud Data Architect, Redeye Apps</h2>
|
||||
<p><em>May 2022 - Present</em>
|
||||
- Greenfields Research, Design and Deployment of S3 datalake (Parquet)
|
||||
- AWS DMS, S3, Athena, Glue
|
||||
- Research Design and Deployment of Catalog (Datahub)
|
||||
- Design of Data Governance Process (Datahub driven)
|
||||
- Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)
|
||||
- CI/CD design and deployment of modelling and orchestration using Gitlab
|
||||
- Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy
|
||||
- Design of ETL/Pipelines (DBT)
|
||||
- Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</p>
|
||||
<h2>Data Engineer, TechConnect IT Solutions</h2>
|
||||
<p><em>August 2021 – May 2022</em>
|
||||
- Design of Cloud Data Batch ETL solutions using Python (Glue)
|
||||
- Design of Cloud Data Streaming ETL solution using Python (Kinesis)
|
||||
- Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs
|
||||
- Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)
|
||||
- Apply Prebuilt ML models to software solutions (Sagemaker)
|
||||
- Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)
|
||||
- API testing and development (gRPC, Rest)</p>
|
||||
<h2>Enterprise Data Warehouse Developer, Auto and General Insurance</h2>
|
||||
<p><em>August 2019 - August 2021</em>
|
||||
- ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL
|
||||
- Bringing new data to the business to analyse for new insights
|
||||
- Redeveloped Version Control and brought git to the data team
|
||||
- Introduced python for API enablement in the Enterprise Data Warehouse
|
||||
- Partnering with the business to focus data project on actual need and translating into technical requirements</p>
|
||||
<h2>Business Analyst, Auto and General Insurance</h2>
|
||||
<p><em>January 2018 - August 2019</em>
|
||||
- Automate Service Performance Reporting using PowerShell/VBA/SAS
|
||||
- Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting
|
||||
- Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends
|
||||
- Technologies used include VBA, PowerShell, SQL, Web API’s, SAS
|
||||
- Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel
|
||||
- Gather Requirements to build meaningful reporting solutions
|
||||
- Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</p>
|
||||
<h2>Forecasting and Capacity Analyst, Auto and General Insurance</h2>
|
||||
<p><em>January 2017 – January 2018</em>
|
||||
- Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers
|
||||
- This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.
|
||||
- Forecast inbound workload requirements for the Auto and General sales call center using time series analysis
|
||||
- Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation
|
||||
- Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy
|
||||
- Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders
|
||||
- Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</p>
|
||||
<h2>Senior HR Performance Analyst, Queensland Department of Justice and Attorney General</h2>
|
||||
<p><em>June 2016 - January 2017</em>
|
||||
- Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics
|
||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
||||
<h2>Workforce Business Analyst, Queensland Department of Justice and Attorney General</h2>
|
||||
<p><em>July 2015 – June 2016</em>
|
||||
- Develop and refine current workforce analysis techniques and databases
|
||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||
- Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit
|
||||
- Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required
|
||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
||||
<h1>EDUCATION</h1>
|
||||
<ul>
|
||||
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
||||
<li>2008 Bachelor of Arts, University of Queensland</li>
|
||||
</ul>
|
||||
<h1>REFERENCES</h1>
|
||||
<ul>
|
||||
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health
|
||||
<em>0428 038 031</em></li>
|
||||
<li>Jaime Brian Head of Cloud Ninjas, TechConnect
|
||||
<em>0422 012 17</em></li>
|
||||
</ul>
|
||||
</article>
|
||||
|
||||
<hr>
|
||||
|
||||
@ -1,5 +1,111 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<feed xmlns="http://www.w3.org/2005/Atom"><title>Andrew Ridgway's Blog</title><link href="http://localhost:8000/" rel="alternate"></link><link href="http://localhost:8000/feeds/all-en.atom.xml" rel="self"></link><id>http://localhost:8000/</id><updated>2024-03-13T20:00:00+10:00</updated><entry><title>A Cover Letter</title><link href="http://localhost:8000/cover-letter.html" rel="alternate"></link><published>2024-02-23T20:00:00+10:00</published><updated>2024-03-13T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2024-02-23:/cover-letter.html</id><summary type="html"><p>A Summary of what I've done and Where I'd like to go for prospective Employers</p></summary><content type="html"><p>To whom it may concern</p>
|
||||
<feed xmlns="http://www.w3.org/2005/Atom"><title>Andrew Ridgway's Blog</title><link href="http://localhost:8000/" rel="alternate"></link><link href="http://localhost:8000/feeds/all-en.atom.xml" rel="self"></link><id>http://localhost:8000/</id><updated>2024-03-13T20:00:00+10:00</updated><entry><title>A Resume</title><link href="http://localhost:8000/cover-letter.html" rel="alternate"></link><published>2024-02-23T20:00:00+10:00</published><updated>2024-03-13T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2024-02-23:/cover-letter.html</id><summary type="html"><p>A Summary of My work Experience</p></summary><content type="html"><h1>OVERVIEW</h1>
|
||||
<p>I am a Senior Data Engineer looking to transition my skills to Data and Solution
|
||||
Architecting as well as project management. I have spent the better part of the
|
||||
last decade refining my abilities in taking business requirements and turning
|
||||
those into actionable data engineering, analytics, and software projects with
|
||||
trackable metrics. I believe in agnosticism when it comes to coding languages
|
||||
and have experimented in my own time with many different languages. In my
|
||||
career I have used Python, .NET, PowerShell, TSQL, VB and SAS (multiple
|
||||
products) in an Enterprise capacity. I also have experience using Google Cloud
|
||||
Platform and AWS tools for ETL and data platform development as well as git
|
||||
for version control and deployment using various IAC tools. I have also
|
||||
conducted data analysis and modelling on business metrics to find relationships
|
||||
between both staff and customer behavior and produced actionable
|
||||
recommendations based on the conclusions. In a private context I have also
|
||||
experimented with C, C# and Kotlin I am looking to further my career by taking
|
||||
my passion for data engineering and analysis as well as web and software
|
||||
development and applying it in a strategic context.</p>
|
||||
<h1>SKILLS &amp; ABILITIES</h1>
|
||||
<ul>
|
||||
<li>Python (scripting, compiling, notebooks – Sagemaker, Jupyter)</li>
|
||||
<li>git</li>
|
||||
<li>SAS (Base, EG, VA)</li>
|
||||
<li>Various Google Cloud Tools (Data Fusion, Compute Engine, Cloud Functions)</li>
|
||||
<li>Various Amazon Tools (EC2, RDS, Kinesis, Glue, Redshift, Lambda, ECS, ECR, EKS)</li>
|
||||
<li>Streaming Technologies (Kafka, Hive, Spark Streaming)</li>
|
||||
<li>Various DB platforms both on Prem and Serverless (MariaDB/MySql,</li>
|
||||
<li>Postgres/Redshift, SQL Server, RDS/Aurora variants)</li>
|
||||
<li>Various Microsoft Products (PowerBI, TSQL, Excel, VBA)</li>
|
||||
<li>Linux Server Administration (cron, bash, systemD)</li>
|
||||
<li>ETL/ELT Development</li>
|
||||
<li>Basic Data Modelling (Kimball, SCD Type 2)</li>
|
||||
<li>IAC (Cloud Formation, Terraform)</li>
|
||||
<li>Datahub Deployment</li>
|
||||
<li>Dagster Orchestration Deployments</li>
|
||||
<li>DBT Modelling and Design Deployments</li>
|
||||
<li>Containerised and Cloud Driven Data Architecture</li>
|
||||
</ul>
|
||||
<h1>EXPERIENCE</h1>
|
||||
<h2>Cloud Data Architect, Redeye Apps</h2>
|
||||
<p><em>May 2022 - Present</em>
|
||||
- Greenfields Research, Design and Deployment of S3 datalake (Parquet)
|
||||
- AWS DMS, S3, Athena, Glue
|
||||
- Research Design and Deployment of Catalog (Datahub)
|
||||
- Design of Data Governance Process (Datahub driven)
|
||||
- Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)
|
||||
- CI/CD design and deployment of modelling and orchestration using Gitlab
|
||||
- Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy
|
||||
- Design of ETL/Pipelines (DBT)
|
||||
- Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</p>
|
||||
<h2>Data Engineer, TechConnect IT Solutions</h2>
|
||||
<p><em>August 2021 – May 2022</em>
|
||||
- Design of Cloud Data Batch ETL solutions using Python (Glue)
|
||||
- Design of Cloud Data Streaming ETL solution using Python (Kinesis)
|
||||
- Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs
|
||||
- Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)
|
||||
- Apply Prebuilt ML models to software solutions (Sagemaker)
|
||||
- Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)
|
||||
- API testing and development (gRPC, Rest)</p>
|
||||
<h2>Enterprise Data Warehouse Developer, Auto and General Insurance</h2>
|
||||
<p><em>August 2019 - August 2021</em>
|
||||
- ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL
|
||||
- Bringing new data to the business to analyse for new insights
|
||||
- Redeveloped Version Control and brought git to the data team
|
||||
- Introduced python for API enablement in the Enterprise Data Warehouse
|
||||
- Partnering with the business to focus data project on actual need and translating into technical requirements</p>
|
||||
<h2>Business Analyst, Auto and General Insurance</h2>
|
||||
<p><em>January 2018 - August 2019</em>
|
||||
- Automate Service Performance Reporting using PowerShell/VBA/SAS
|
||||
- Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting
|
||||
- Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends
|
||||
- Technologies used include VBA, PowerShell, SQL, Web API’s, SAS
|
||||
- Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel
|
||||
- Gather Requirements to build meaningful reporting solutions
|
||||
- Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</p>
|
||||
<h2>Forecasting and Capacity Analyst, Auto and General Insurance</h2>
|
||||
<p><em>January 2017 – January 2018</em>
|
||||
- Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers
|
||||
- This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.
|
||||
- Forecast inbound workload requirements for the Auto and General sales call center using time series analysis
|
||||
- Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation
|
||||
- Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy
|
||||
- Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders
|
||||
- Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</p>
|
||||
<h2>Senior HR Performance Analyst, Queensland Department of Justice and Attorney General</h2>
|
||||
<p><em>June 2016 - January 2017</em>
|
||||
- Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics
|
||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
||||
<h2>Workforce Business Analyst, Queensland Department of Justice and Attorney General</h2>
|
||||
<p><em>July 2015 – June 2016</em>
|
||||
- Develop and refine current workforce analysis techniques and databases
|
||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||
- Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit
|
||||
- Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required
|
||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
||||
<h1>EDUCATION</h1>
|
||||
<ul>
|
||||
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
||||
<li>2008 Bachelor of Arts, University of Queensland</li>
|
||||
</ul>
|
||||
<h1>REFERENCES</h1>
|
||||
<ul>
|
||||
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health
|
||||
<em>0428 038 031</em></li>
|
||||
<li>Jaime Brian Head of Cloud Ninjas, TechConnect
|
||||
<em>0422 012 17</em></li>
|
||||
</ul></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry><entry><title>A Cover Letter</title><link href="http://localhost:8000/cover-letter.html" rel="alternate"></link><published>2024-02-23T20:00:00+10:00</published><updated>2024-03-13T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2024-02-23:/cover-letter.html</id><summary type="html"><p>A Summary of what I've done and Where I'd like to go for prospective Employers</p></summary><content type="html"><p>To whom it may concern</p>
|
||||
<p>My name is Andrew Ridgway and I am a Data and Technology professional looking to embark on the next step in my career.</p>
|
||||
<p>I have over 10 years’ experience in System and Data Architecture, Data Modelling and Orchestration, Business and Technical Analysis and System and Development Process Design. Most of this has been in developing Cloud architectures and workloads on AWS and GCP Including ML workloads using Sagemaker. </p>
|
||||
<p>In my current role I have Proposed, Designed and built the data platform currently used by business. This includes internal and external data products as well as the infrastructure and modelling to support these. This role has seen me liaise with stakeholders of all levels of the business from Analysts in the Customer Experience team right up to C suite executives and preparing material for board members. I understand the complexity of communicating complex system design to different level stakeholders and the complexities of involved in communicating to both technical and less technical employees particularly in relation to data and ML technologies. </p>
|
||||
|
||||
@ -1,5 +1,111 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<feed xmlns="http://www.w3.org/2005/Atom"><title>Andrew Ridgway's Blog</title><link href="http://localhost:8000/" rel="alternate"></link><link href="http://localhost:8000/feeds/all.atom.xml" rel="self"></link><id>http://localhost:8000/</id><updated>2024-03-13T20:00:00+10:00</updated><entry><title>A Cover Letter</title><link href="http://localhost:8000/cover-letter.html" rel="alternate"></link><published>2024-02-23T20:00:00+10:00</published><updated>2024-03-13T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2024-02-23:/cover-letter.html</id><summary type="html"><p>A Summary of what I've done and Where I'd like to go for prospective Employers</p></summary><content type="html"><p>To whom it may concern</p>
|
||||
<feed xmlns="http://www.w3.org/2005/Atom"><title>Andrew Ridgway's Blog</title><link href="http://localhost:8000/" rel="alternate"></link><link href="http://localhost:8000/feeds/all.atom.xml" rel="self"></link><id>http://localhost:8000/</id><updated>2024-03-13T20:00:00+10:00</updated><entry><title>A Resume</title><link href="http://localhost:8000/cover-letter.html" rel="alternate"></link><published>2024-02-23T20:00:00+10:00</published><updated>2024-03-13T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2024-02-23:/cover-letter.html</id><summary type="html"><p>A Summary of My work Experience</p></summary><content type="html"><h1>OVERVIEW</h1>
|
||||
<p>I am a Senior Data Engineer looking to transition my skills to Data and Solution
|
||||
Architecting as well as project management. I have spent the better part of the
|
||||
last decade refining my abilities in taking business requirements and turning
|
||||
those into actionable data engineering, analytics, and software projects with
|
||||
trackable metrics. I believe in agnosticism when it comes to coding languages
|
||||
and have experimented in my own time with many different languages. In my
|
||||
career I have used Python, .NET, PowerShell, TSQL, VB and SAS (multiple
|
||||
products) in an Enterprise capacity. I also have experience using Google Cloud
|
||||
Platform and AWS tools for ETL and data platform development as well as git
|
||||
for version control and deployment using various IAC tools. I have also
|
||||
conducted data analysis and modelling on business metrics to find relationships
|
||||
between both staff and customer behavior and produced actionable
|
||||
recommendations based on the conclusions. In a private context I have also
|
||||
experimented with C, C# and Kotlin I am looking to further my career by taking
|
||||
my passion for data engineering and analysis as well as web and software
|
||||
development and applying it in a strategic context.</p>
|
||||
<h1>SKILLS &amp; ABILITIES</h1>
|
||||
<ul>
|
||||
<li>Python (scripting, compiling, notebooks – Sagemaker, Jupyter)</li>
|
||||
<li>git</li>
|
||||
<li>SAS (Base, EG, VA)</li>
|
||||
<li>Various Google Cloud Tools (Data Fusion, Compute Engine, Cloud Functions)</li>
|
||||
<li>Various Amazon Tools (EC2, RDS, Kinesis, Glue, Redshift, Lambda, ECS, ECR, EKS)</li>
|
||||
<li>Streaming Technologies (Kafka, Hive, Spark Streaming)</li>
|
||||
<li>Various DB platforms both on Prem and Serverless (MariaDB/MySql,</li>
|
||||
<li>Postgres/Redshift, SQL Server, RDS/Aurora variants)</li>
|
||||
<li>Various Microsoft Products (PowerBI, TSQL, Excel, VBA)</li>
|
||||
<li>Linux Server Administration (cron, bash, systemD)</li>
|
||||
<li>ETL/ELT Development</li>
|
||||
<li>Basic Data Modelling (Kimball, SCD Type 2)</li>
|
||||
<li>IAC (Cloud Formation, Terraform)</li>
|
||||
<li>Datahub Deployment</li>
|
||||
<li>Dagster Orchestration Deployments</li>
|
||||
<li>DBT Modelling and Design Deployments</li>
|
||||
<li>Containerised and Cloud Driven Data Architecture</li>
|
||||
</ul>
|
||||
<h1>EXPERIENCE</h1>
|
||||
<h2>Cloud Data Architect, Redeye Apps</h2>
|
||||
<p><em>May 2022 - Present</em>
|
||||
- Greenfields Research, Design and Deployment of S3 datalake (Parquet)
|
||||
- AWS DMS, S3, Athena, Glue
|
||||
- Research Design and Deployment of Catalog (Datahub)
|
||||
- Design of Data Governance Process (Datahub driven)
|
||||
- Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)
|
||||
- CI/CD design and deployment of modelling and orchestration using Gitlab
|
||||
- Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy
|
||||
- Design of ETL/Pipelines (DBT)
|
||||
- Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</p>
|
||||
<h2>Data Engineer, TechConnect IT Solutions</h2>
|
||||
<p><em>August 2021 – May 2022</em>
|
||||
- Design of Cloud Data Batch ETL solutions using Python (Glue)
|
||||
- Design of Cloud Data Streaming ETL solution using Python (Kinesis)
|
||||
- Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs
|
||||
- Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)
|
||||
- Apply Prebuilt ML models to software solutions (Sagemaker)
|
||||
- Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)
|
||||
- API testing and development (gRPC, Rest)</p>
|
||||
<h2>Enterprise Data Warehouse Developer, Auto and General Insurance</h2>
|
||||
<p><em>August 2019 - August 2021</em>
|
||||
- ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL
|
||||
- Bringing new data to the business to analyse for new insights
|
||||
- Redeveloped Version Control and brought git to the data team
|
||||
- Introduced python for API enablement in the Enterprise Data Warehouse
|
||||
- Partnering with the business to focus data project on actual need and translating into technical requirements</p>
|
||||
<h2>Business Analyst, Auto and General Insurance</h2>
|
||||
<p><em>January 2018 - August 2019</em>
|
||||
- Automate Service Performance Reporting using PowerShell/VBA/SAS
|
||||
- Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting
|
||||
- Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends
|
||||
- Technologies used include VBA, PowerShell, SQL, Web API’s, SAS
|
||||
- Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel
|
||||
- Gather Requirements to build meaningful reporting solutions
|
||||
- Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</p>
|
||||
<h2>Forecasting and Capacity Analyst, Auto and General Insurance</h2>
|
||||
<p><em>January 2017 – January 2018</em>
|
||||
- Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers
|
||||
- This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.
|
||||
- Forecast inbound workload requirements for the Auto and General sales call center using time series analysis
|
||||
- Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation
|
||||
- Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy
|
||||
- Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders
|
||||
- Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</p>
|
||||
<h2>Senior HR Performance Analyst, Queensland Department of Justice and Attorney General</h2>
|
||||
<p><em>June 2016 - January 2017</em>
|
||||
- Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics
|
||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
||||
<h2>Workforce Business Analyst, Queensland Department of Justice and Attorney General</h2>
|
||||
<p><em>July 2015 – June 2016</em>
|
||||
- Develop and refine current workforce analysis techniques and databases
|
||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||
- Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit
|
||||
- Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required
|
||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
||||
<h1>EDUCATION</h1>
|
||||
<ul>
|
||||
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
||||
<li>2008 Bachelor of Arts, University of Queensland</li>
|
||||
</ul>
|
||||
<h1>REFERENCES</h1>
|
||||
<ul>
|
||||
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health
|
||||
<em>0428 038 031</em></li>
|
||||
<li>Jaime Brian Head of Cloud Ninjas, TechConnect
|
||||
<em>0422 012 17</em></li>
|
||||
</ul></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry><entry><title>A Cover Letter</title><link href="http://localhost:8000/cover-letter.html" rel="alternate"></link><published>2024-02-23T20:00:00+10:00</published><updated>2024-03-13T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2024-02-23:/cover-letter.html</id><summary type="html"><p>A Summary of what I've done and Where I'd like to go for prospective Employers</p></summary><content type="html"><p>To whom it may concern</p>
|
||||
<p>My name is Andrew Ridgway and I am a Data and Technology professional looking to embark on the next step in my career.</p>
|
||||
<p>I have over 10 years’ experience in System and Data Architecture, Data Modelling and Orchestration, Business and Technical Analysis and System and Development Process Design. Most of this has been in developing Cloud architectures and workloads on AWS and GCP Including ML workloads using Sagemaker. </p>
|
||||
<p>In my current role I have Proposed, Designed and built the data platform currently used by business. This includes internal and external data products as well as the infrastructure and modelling to support these. This role has seen me liaise with stakeholders of all levels of the business from Analysts in the Customer Experience team right up to C suite executives and preparing material for board members. I understand the complexity of communicating complex system design to different level stakeholders and the complexities of involved in communicating to both technical and less technical employees particularly in relation to data and ML technologies. </p>
|
||||
@ -11,7 +117,125 @@
|
||||
<p>I look forward to hearing from you soon.</p>
|
||||
<p>Sincerely,</p>
|
||||
<hr>
|
||||
<p>Andrew Ridgway</p></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry><entry><title>Metabase and DuckDB</title><link href="http://localhost:8000/metabase-duckdb.html" rel="alternate"></link><published>2023-11-15T20:00:00+10:00</published><updated>2023-11-15T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2023-11-15:/metabase-duckdb.html</id><summary type="html"><p>Using Metabase and DuckDB to create an embedded Reporting Container bringing the data as close to the report as possible</p></summary><content type="html"><p>Ahhhh <a href="https://duckdb.org/">DuckDB</a> if you're even partly floating around in the data space you've probably been hearing ALOT about it and it's <em>"Datawarehouse on your laptop"</em> mantra. However, the OTHER application that sometimes gets missed is <em>"SQLite for OLAP workloads"</em> and it was this concept that once I grasped it gave me a very interesting idea.... What if we could take the very pretty Aggregate Layer of our Data(warehouse/LakeHouse/Lake) and put that data right next to presentation layer of the lake, reducing network latency and... hopefully... have presentation reports running over very large workloads in the blink of an eye. It might even be fast enough that it could be deployed and embedded </p>
|
||||
<p>Andrew Ridgway</p></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry><entry><title>A Cover Letter</title><link href="http://localhost:8000/cover-letter.html" rel="alternate"></link><published>2024-02-23T20:00:00+10:00</published><updated>2024-03-13T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2024-02-23:/cover-letter.html</id><summary type="html"><p>A Summary of what I've done and Where I'd like to go for prospective Employers</p></summary><content type="html"><p>To whom it may concern</p>
|
||||
<p>My name is Andrew Ridgway and I am a Data and Technology professional looking to embark on the next step in my career.</p>
|
||||
<p>I have over 10 years’ experience in System and Data Architecture, Data Modelling and Orchestration, Business and Technical Analysis and System and Development Process Design. Most of this has been in developing Cloud architectures and workloads on AWS and GCP Including ML workloads using Sagemaker. </p>
|
||||
<p>In my current role I have Proposed, Designed and built the data platform currently used by business. This includes internal and external data products as well as the infrastructure and modelling to support these. This role has seen me liaise with stakeholders of all levels of the business from Analysts in the Customer Experience team right up to C suite executives and preparing material for board members. I understand the complexity of communicating complex system design to different level stakeholders and the complexities of involved in communicating to both technical and less technical employees particularly in relation to data and ML technologies. </p>
|
||||
<p>I have also worked as a technical consultant to many businesses and have assisted with the design and implementation of systems for a wide range of industries including financial services, mining and retail. I understand the complexities created by regulation in these environments and understand that this can sometimes necessitate the use of technologies and designs, including legacy systems and designs, I wouldn’t normally use. I also have a passion of designing systems that enable these organisations to realise the benefits of CI/CD on workloads they would not traditionally use this capability. In particular I took a very traditional legacy Data Warehousing team and implemented a solution that meant version control was no longer controlled by a daily copy and paste of folders with dates on major updates. My solution involved establishing guidelines of use of git version control so that this could happen automatically as people committed new code to the core code base. As I have moved into cloud architecture I have made sure to use best practice and ensure everything I build isn’t considered production ready until it is in IAC and deployed through a CI/CD pipeline.</p>
|
||||
<p>In a personal capacity I am an avid tech and ML enthusiast. I have designed my own cluster including monitoring and deployment that runs several services that my family uses including chat and DNS and am in the process of designing a “set and forget” system that will allows me to have multi user tenancies on hardware I operate that should enable us to have the niceties of cloud services like email, storage and scheduling with the safety of knowing where that data is stored and exactly how it is used. I also like to design small IoT devices out of Arduino boards allowing me to monitor and control different facets of our house like temperature and light. </p>
|
||||
<p>Currently I am working on a project to merge my skill in SQL Modelling and Orchestration with GPT API’s to try and lessen that burden. You can see some of this work in its very early stages here:
|
||||
(gpt-sql-generator)[https://github.com/armistace/gpt-sql-generator]
|
||||
(dbt_sources_generator)[https://github.com/armistace/datahub_dbt_sources_generator]</p>
|
||||
<p>I look forward to hearing from you soon.</p>
|
||||
<p>Sincerely,</p>
|
||||
<hr>
|
||||
<p>Andrew Ridgway</p></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry><entry><title>A Resume</title><link href="http://localhost:8000/cover-letter.html" rel="alternate"></link><published>2024-02-23T20:00:00+10:00</published><updated>2024-03-13T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2024-02-23:/cover-letter.html</id><summary type="html"><p>A Summary of My work Experience</p></summary><content type="html"><h1>OVERVIEW</h1>
|
||||
<p>I am a Senior Data Engineer looking to transition my skills to Data and Solution
|
||||
Architecting as well as project management. I have spent the better part of the
|
||||
last decade refining my abilities in taking business requirements and turning
|
||||
those into actionable data engineering, analytics, and software projects with
|
||||
trackable metrics. I believe in agnosticism when it comes to coding languages
|
||||
and have experimented in my own time with many different languages. In my
|
||||
career I have used Python, .NET, PowerShell, TSQL, VB and SAS (multiple
|
||||
products) in an Enterprise capacity. I also have experience using Google Cloud
|
||||
Platform and AWS tools for ETL and data platform development as well as git
|
||||
for version control and deployment using various IAC tools. I have also
|
||||
conducted data analysis and modelling on business metrics to find relationships
|
||||
between both staff and customer behavior and produced actionable
|
||||
recommendations based on the conclusions. In a private context I have also
|
||||
experimented with C, C# and Kotlin I am looking to further my career by taking
|
||||
my passion for data engineering and analysis as well as web and software
|
||||
development and applying it in a strategic context.</p>
|
||||
<h1>SKILLS &amp; ABILITIES</h1>
|
||||
<ul>
|
||||
<li>Python (scripting, compiling, notebooks – Sagemaker, Jupyter)</li>
|
||||
<li>git</li>
|
||||
<li>SAS (Base, EG, VA)</li>
|
||||
<li>Various Google Cloud Tools (Data Fusion, Compute Engine, Cloud Functions)</li>
|
||||
<li>Various Amazon Tools (EC2, RDS, Kinesis, Glue, Redshift, Lambda, ECS, ECR, EKS)</li>
|
||||
<li>Streaming Technologies (Kafka, Hive, Spark Streaming)</li>
|
||||
<li>Various DB platforms both on Prem and Serverless (MariaDB/MySql,</li>
|
||||
<li>Postgres/Redshift, SQL Server, RDS/Aurora variants)</li>
|
||||
<li>Various Microsoft Products (PowerBI, TSQL, Excel, VBA)</li>
|
||||
<li>Linux Server Administration (cron, bash, systemD)</li>
|
||||
<li>ETL/ELT Development</li>
|
||||
<li>Basic Data Modelling (Kimball, SCD Type 2)</li>
|
||||
<li>IAC (Cloud Formation, Terraform)</li>
|
||||
<li>Datahub Deployment</li>
|
||||
<li>Dagster Orchestration Deployments</li>
|
||||
<li>DBT Modelling and Design Deployments</li>
|
||||
<li>Containerised and Cloud Driven Data Architecture</li>
|
||||
</ul>
|
||||
<h1>EXPERIENCE</h1>
|
||||
<h2>Cloud Data Architect, Redeye Apps</h2>
|
||||
<p><em>May 2022 - Present</em>
|
||||
- Greenfields Research, Design and Deployment of S3 datalake (Parquet)
|
||||
- AWS DMS, S3, Athena, Glue
|
||||
- Research Design and Deployment of Catalog (Datahub)
|
||||
- Design of Data Governance Process (Datahub driven)
|
||||
- Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)
|
||||
- CI/CD design and deployment of modelling and orchestration using Gitlab
|
||||
- Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy
|
||||
- Design of ETL/Pipelines (DBT)
|
||||
- Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</p>
|
||||
<h2>Data Engineer, TechConnect IT Solutions</h2>
|
||||
<p><em>August 2021 – May 2022</em>
|
||||
- Design of Cloud Data Batch ETL solutions using Python (Glue)
|
||||
- Design of Cloud Data Streaming ETL solution using Python (Kinesis)
|
||||
- Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs
|
||||
- Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)
|
||||
- Apply Prebuilt ML models to software solutions (Sagemaker)
|
||||
- Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)
|
||||
- API testing and development (gRPC, Rest)</p>
|
||||
<h2>Enterprise Data Warehouse Developer, Auto and General Insurance</h2>
|
||||
<p><em>August 2019 - August 2021</em>
|
||||
- ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL
|
||||
- Bringing new data to the business to analyse for new insights
|
||||
- Redeveloped Version Control and brought git to the data team
|
||||
- Introduced python for API enablement in the Enterprise Data Warehouse
|
||||
- Partnering with the business to focus data project on actual need and translating into technical requirements</p>
|
||||
<h2>Business Analyst, Auto and General Insurance</h2>
|
||||
<p><em>January 2018 - August 2019</em>
|
||||
- Automate Service Performance Reporting using PowerShell/VBA/SAS
|
||||
- Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting
|
||||
- Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends
|
||||
- Technologies used include VBA, PowerShell, SQL, Web API’s, SAS
|
||||
- Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel
|
||||
- Gather Requirements to build meaningful reporting solutions
|
||||
- Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</p>
|
||||
<h2>Forecasting and Capacity Analyst, Auto and General Insurance</h2>
|
||||
<p><em>January 2017 – January 2018</em>
|
||||
- Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers
|
||||
- This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.
|
||||
- Forecast inbound workload requirements for the Auto and General sales call center using time series analysis
|
||||
- Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation
|
||||
- Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy
|
||||
- Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders
|
||||
- Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</p>
|
||||
<h2>Senior HR Performance Analyst, Queensland Department of Justice and Attorney General</h2>
|
||||
<p><em>June 2016 - January 2017</em>
|
||||
- Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics
|
||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
||||
<h2>Workforce Business Analyst, Queensland Department of Justice and Attorney General</h2>
|
||||
<p><em>July 2015 – June 2016</em>
|
||||
- Develop and refine current workforce analysis techniques and databases
|
||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||
- Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit
|
||||
- Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required
|
||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
||||
<h1>EDUCATION</h1>
|
||||
<ul>
|
||||
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
||||
<li>2008 Bachelor of Arts, University of Queensland</li>
|
||||
</ul>
|
||||
<h1>REFERENCES</h1>
|
||||
<ul>
|
||||
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health
|
||||
<em>0428 038 031</em></li>
|
||||
<li>Jaime Brian Head of Cloud Ninjas, TechConnect
|
||||
<em>0422 012 17</em></li>
|
||||
</ul></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry><entry><title>Metabase and DuckDB</title><link href="http://localhost:8000/metabase-duckdb.html" rel="alternate"></link><published>2023-11-15T20:00:00+10:00</published><updated>2023-11-15T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2023-11-15:/metabase-duckdb.html</id><summary type="html"><p>Using Metabase and DuckDB to create an embedded Reporting Container bringing the data as close to the report as possible</p></summary><content type="html"><p>Ahhhh <a href="https://duckdb.org/">DuckDB</a> if you're even partly floating around in the data space you've probably been hearing ALOT about it and it's <em>"Datawarehouse on your laptop"</em> mantra. However, the OTHER application that sometimes gets missed is <em>"SQLite for OLAP workloads"</em> and it was this concept that once I grasped it gave me a very interesting idea.... What if we could take the very pretty Aggregate Layer of our Data(warehouse/LakeHouse/Lake) and put that data right next to presentation layer of the lake, reducing network latency and... hopefully... have presentation reports running over very large workloads in the blink of an eye. It might even be fast enough that it could be deployed and embedded </p>
|
||||
<p>However, for this to work we need some form of conatinerised reporting application.... lucky for us there is <a href="https://www.metabase.com/">Metabase</a> which is a fantastic little reporting application that has an open core. So this got me thinking... Can I put these two applications together and create a Reporting Layer with report embedding capabilities that is deployable in the cluster and has a admin UI accesible over a web page all whilst keeping the data locked to our network?</p>
|
||||
<h3>The Beginnings of an Idea</h3>
|
||||
<p>Ok so... Big first question. Can Duckdb and Metabase talk? Well... not quite. But first lets take a quick look at the architecture we'll be employing here </p>
|
||||
|
||||
@ -1,5 +1,111 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<feed xmlns="http://www.w3.org/2005/Atom"><title>Andrew Ridgway's Blog - Andrew Ridgway</title><link href="http://localhost:8000/" rel="alternate"></link><link href="http://localhost:8000/feeds/andrew-ridgway.atom.xml" rel="self"></link><id>http://localhost:8000/</id><updated>2024-03-13T20:00:00+10:00</updated><entry><title>A Cover Letter</title><link href="http://localhost:8000/cover-letter.html" rel="alternate"></link><published>2024-02-23T20:00:00+10:00</published><updated>2024-03-13T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2024-02-23:/cover-letter.html</id><summary type="html"><p>A Summary of what I've done and Where I'd like to go for prospective Employers</p></summary><content type="html"><p>To whom it may concern</p>
|
||||
<feed xmlns="http://www.w3.org/2005/Atom"><title>Andrew Ridgway's Blog - Andrew Ridgway</title><link href="http://localhost:8000/" rel="alternate"></link><link href="http://localhost:8000/feeds/andrew-ridgway.atom.xml" rel="self"></link><id>http://localhost:8000/</id><updated>2024-03-13T20:00:00+10:00</updated><entry><title>A Resume</title><link href="http://localhost:8000/cover-letter.html" rel="alternate"></link><published>2024-02-23T20:00:00+10:00</published><updated>2024-03-13T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2024-02-23:/cover-letter.html</id><summary type="html"><p>A Summary of My work Experience</p></summary><content type="html"><h1>OVERVIEW</h1>
|
||||
<p>I am a Senior Data Engineer looking to transition my skills to Data and Solution
|
||||
Architecting as well as project management. I have spent the better part of the
|
||||
last decade refining my abilities in taking business requirements and turning
|
||||
those into actionable data engineering, analytics, and software projects with
|
||||
trackable metrics. I believe in agnosticism when it comes to coding languages
|
||||
and have experimented in my own time with many different languages. In my
|
||||
career I have used Python, .NET, PowerShell, TSQL, VB and SAS (multiple
|
||||
products) in an Enterprise capacity. I also have experience using Google Cloud
|
||||
Platform and AWS tools for ETL and data platform development as well as git
|
||||
for version control and deployment using various IAC tools. I have also
|
||||
conducted data analysis and modelling on business metrics to find relationships
|
||||
between both staff and customer behavior and produced actionable
|
||||
recommendations based on the conclusions. In a private context I have also
|
||||
experimented with C, C# and Kotlin I am looking to further my career by taking
|
||||
my passion for data engineering and analysis as well as web and software
|
||||
development and applying it in a strategic context.</p>
|
||||
<h1>SKILLS &amp; ABILITIES</h1>
|
||||
<ul>
|
||||
<li>Python (scripting, compiling, notebooks – Sagemaker, Jupyter)</li>
|
||||
<li>git</li>
|
||||
<li>SAS (Base, EG, VA)</li>
|
||||
<li>Various Google Cloud Tools (Data Fusion, Compute Engine, Cloud Functions)</li>
|
||||
<li>Various Amazon Tools (EC2, RDS, Kinesis, Glue, Redshift, Lambda, ECS, ECR, EKS)</li>
|
||||
<li>Streaming Technologies (Kafka, Hive, Spark Streaming)</li>
|
||||
<li>Various DB platforms both on Prem and Serverless (MariaDB/MySql,</li>
|
||||
<li>Postgres/Redshift, SQL Server, RDS/Aurora variants)</li>
|
||||
<li>Various Microsoft Products (PowerBI, TSQL, Excel, VBA)</li>
|
||||
<li>Linux Server Administration (cron, bash, systemD)</li>
|
||||
<li>ETL/ELT Development</li>
|
||||
<li>Basic Data Modelling (Kimball, SCD Type 2)</li>
|
||||
<li>IAC (Cloud Formation, Terraform)</li>
|
||||
<li>Datahub Deployment</li>
|
||||
<li>Dagster Orchestration Deployments</li>
|
||||
<li>DBT Modelling and Design Deployments</li>
|
||||
<li>Containerised and Cloud Driven Data Architecture</li>
|
||||
</ul>
|
||||
<h1>EXPERIENCE</h1>
|
||||
<h2>Cloud Data Architect, Redeye Apps</h2>
|
||||
<p><em>May 2022 - Present</em>
|
||||
- Greenfields Research, Design and Deployment of S3 datalake (Parquet)
|
||||
- AWS DMS, S3, Athena, Glue
|
||||
- Research Design and Deployment of Catalog (Datahub)
|
||||
- Design of Data Governance Process (Datahub driven)
|
||||
- Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)
|
||||
- CI/CD design and deployment of modelling and orchestration using Gitlab
|
||||
- Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy
|
||||
- Design of ETL/Pipelines (DBT)
|
||||
- Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</p>
|
||||
<h2>Data Engineer, TechConnect IT Solutions</h2>
|
||||
<p><em>August 2021 – May 2022</em>
|
||||
- Design of Cloud Data Batch ETL solutions using Python (Glue)
|
||||
- Design of Cloud Data Streaming ETL solution using Python (Kinesis)
|
||||
- Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs
|
||||
- Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)
|
||||
- Apply Prebuilt ML models to software solutions (Sagemaker)
|
||||
- Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)
|
||||
- API testing and development (gRPC, Rest)</p>
|
||||
<h2>Enterprise Data Warehouse Developer, Auto and General Insurance</h2>
|
||||
<p><em>August 2019 - August 2021</em>
|
||||
- ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL
|
||||
- Bringing new data to the business to analyse for new insights
|
||||
- Redeveloped Version Control and brought git to the data team
|
||||
- Introduced python for API enablement in the Enterprise Data Warehouse
|
||||
- Partnering with the business to focus data project on actual need and translating into technical requirements</p>
|
||||
<h2>Business Analyst, Auto and General Insurance</h2>
|
||||
<p><em>January 2018 - August 2019</em>
|
||||
- Automate Service Performance Reporting using PowerShell/VBA/SAS
|
||||
- Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting
|
||||
- Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends
|
||||
- Technologies used include VBA, PowerShell, SQL, Web API’s, SAS
|
||||
- Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel
|
||||
- Gather Requirements to build meaningful reporting solutions
|
||||
- Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</p>
|
||||
<h2>Forecasting and Capacity Analyst, Auto and General Insurance</h2>
|
||||
<p><em>January 2017 – January 2018</em>
|
||||
- Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers
|
||||
- This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.
|
||||
- Forecast inbound workload requirements for the Auto and General sales call center using time series analysis
|
||||
- Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation
|
||||
- Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy
|
||||
- Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders
|
||||
- Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</p>
|
||||
<h2>Senior HR Performance Analyst, Queensland Department of Justice and Attorney General</h2>
|
||||
<p><em>June 2016 - January 2017</em>
|
||||
- Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics
|
||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
||||
<h2>Workforce Business Analyst, Queensland Department of Justice and Attorney General</h2>
|
||||
<p><em>July 2015 – June 2016</em>
|
||||
- Develop and refine current workforce analysis techniques and databases
|
||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||
- Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit
|
||||
- Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required
|
||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
||||
<h1>EDUCATION</h1>
|
||||
<ul>
|
||||
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
||||
<li>2008 Bachelor of Arts, University of Queensland</li>
|
||||
</ul>
|
||||
<h1>REFERENCES</h1>
|
||||
<ul>
|
||||
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health
|
||||
<em>0428 038 031</em></li>
|
||||
<li>Jaime Brian Head of Cloud Ninjas, TechConnect
|
||||
<em>0422 012 17</em></li>
|
||||
</ul></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry><entry><title>A Cover Letter</title><link href="http://localhost:8000/cover-letter.html" rel="alternate"></link><published>2024-02-23T20:00:00+10:00</published><updated>2024-03-13T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2024-02-23:/cover-letter.html</id><summary type="html"><p>A Summary of what I've done and Where I'd like to go for prospective Employers</p></summary><content type="html"><p>To whom it may concern</p>
|
||||
<p>My name is Andrew Ridgway and I am a Data and Technology professional looking to embark on the next step in my career.</p>
|
||||
<p>I have over 10 years’ experience in System and Data Architecture, Data Modelling and Orchestration, Business and Technical Analysis and System and Development Process Design. Most of this has been in developing Cloud architectures and workloads on AWS and GCP Including ML workloads using Sagemaker. </p>
|
||||
<p>In my current role I have Proposed, Designed and built the data platform currently used by business. This includes internal and external data products as well as the infrastructure and modelling to support these. This role has seen me liaise with stakeholders of all levels of the business from Analysts in the Customer Experience team right up to C suite executives and preparing material for board members. I understand the complexity of communicating complex system design to different level stakeholders and the complexities of involved in communicating to both technical and less technical employees particularly in relation to data and ML technologies. </p>
|
||||
|
||||
@ -1,2 +1,2 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<rss version="2.0"><channel><title>Andrew Ridgway's Blog - Andrew Ridgway</title><link>http://localhost:8000/</link><description></description><lastBuildDate>Wed, 13 Mar 2024 20:00:00 +1000</lastBuildDate><item><title>A Cover Letter</title><link>http://localhost:8000/cover-letter.html</link><description><p>A Summary of what I've done and Where I'd like to go for prospective Employers</p></description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Andrew Ridgway</dc:creator><pubDate>Fri, 23 Feb 2024 20:00:00 +1000</pubDate><guid isPermaLink="false">tag:localhost,2024-02-23:/cover-letter.html</guid><category>Resume</category><category>Cover Letter</category><category>Resume</category></item><item><title>Metabase and DuckDB</title><link>http://localhost:8000/metabase-duckdb.html</link><description><p>Using Metabase and DuckDB to create an embedded Reporting Container bringing the data as close to the report as possible</p></description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Andrew Ridgway</dc:creator><pubDate>Wed, 15 Nov 2023 20:00:00 +1000</pubDate><guid isPermaLink="false">tag:localhost,2023-11-15:/metabase-duckdb.html</guid><category>Business Intelligence</category><category>data engineering</category><category>Metabase</category><category>DuckDB</category><category>embedded</category></item><item><title>Implmenting Appflow in a Production Datalake</title><link>http://localhost:8000/appflow-production.html</link><description><p>How Appflow simplified a major extract layer and when I choose Managed Services</p></description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Andrew Ridgway</dc:creator><pubDate>Tue, 23 May 2023 20:00:00 +1000</pubDate><guid isPermaLink="false">tag:localhost,2023-05-23:/appflow-production.html</guid><category>Data Engineering</category><category>data engineering</category><category>Amazon</category><category>Managed Services</category></item><item><title>Dawn of another blog attempt</title><link>http://localhost:8000/how-i-built-the-damn-thing.html</link><description><p>Containers and How I take my learnings from home and apply them to work</p></description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Andrew Ridgway</dc:creator><pubDate>Wed, 10 May 2023 20:00:00 +1000</pubDate><guid isPermaLink="false">tag:localhost,2023-05-10:/how-i-built-the-damn-thing.html</guid><category>Data Engineering</category><category>data engineering</category><category>containers</category></item></channel></rss>
|
||||
<rss version="2.0"><channel><title>Andrew Ridgway's Blog - Andrew Ridgway</title><link>http://localhost:8000/</link><description></description><lastBuildDate>Wed, 13 Mar 2024 20:00:00 +1000</lastBuildDate><item><title>A Resume</title><link>http://localhost:8000/cover-letter.html</link><description><p>A Summary of My work Experience</p></description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Andrew Ridgway</dc:creator><pubDate>Fri, 23 Feb 2024 20:00:00 +1000</pubDate><guid isPermaLink="false">tag:localhost,2024-02-23:/cover-letter.html</guid><category>Resume</category><category>Cover Letter</category><category>Resume</category></item><item><title>A Cover Letter</title><link>http://localhost:8000/cover-letter.html</link><description><p>A Summary of what I've done and Where I'd like to go for prospective Employers</p></description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Andrew Ridgway</dc:creator><pubDate>Fri, 23 Feb 2024 20:00:00 +1000</pubDate><guid isPermaLink="false">tag:localhost,2024-02-23:/cover-letter.html</guid><category>Resume</category><category>Cover Letter</category><category>Resume</category></item><item><title>Metabase and DuckDB</title><link>http://localhost:8000/metabase-duckdb.html</link><description><p>Using Metabase and DuckDB to create an embedded Reporting Container bringing the data as close to the report as possible</p></description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Andrew Ridgway</dc:creator><pubDate>Wed, 15 Nov 2023 20:00:00 +1000</pubDate><guid isPermaLink="false">tag:localhost,2023-11-15:/metabase-duckdb.html</guid><category>Business Intelligence</category><category>data engineering</category><category>Metabase</category><category>DuckDB</category><category>embedded</category></item><item><title>Implmenting Appflow in a Production Datalake</title><link>http://localhost:8000/appflow-production.html</link><description><p>How Appflow simplified a major extract layer and when I choose Managed Services</p></description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Andrew Ridgway</dc:creator><pubDate>Tue, 23 May 2023 20:00:00 +1000</pubDate><guid isPermaLink="false">tag:localhost,2023-05-23:/appflow-production.html</guid><category>Data Engineering</category><category>data engineering</category><category>Amazon</category><category>Managed Services</category></item><item><title>Dawn of another blog attempt</title><link>http://localhost:8000/how-i-built-the-damn-thing.html</link><description><p>Containers and How I take my learnings from home and apply them to work</p></description><dc:creator xmlns:dc="http://purl.org/dc/elements/1.1/">Andrew Ridgway</dc:creator><pubDate>Wed, 10 May 2023 20:00:00 +1000</pubDate><guid isPermaLink="false">tag:localhost,2023-05-10:/how-i-built-the-damn-thing.html</guid><category>Data Engineering</category><category>data engineering</category><category>containers</category></item></channel></rss>
|
||||
@ -1,5 +1,111 @@
|
||||
<?xml version="1.0" encoding="utf-8"?>
|
||||
<feed xmlns="http://www.w3.org/2005/Atom"><title>Andrew Ridgway's Blog - Resume</title><link href="http://localhost:8000/" rel="alternate"></link><link href="http://localhost:8000/feeds/resume.atom.xml" rel="self"></link><id>http://localhost:8000/</id><updated>2024-03-13T20:00:00+10:00</updated><entry><title>A Cover Letter</title><link href="http://localhost:8000/cover-letter.html" rel="alternate"></link><published>2024-02-23T20:00:00+10:00</published><updated>2024-03-13T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2024-02-23:/cover-letter.html</id><summary type="html"><p>A Summary of what I've done and Where I'd like to go for prospective Employers</p></summary><content type="html"><p>To whom it may concern</p>
|
||||
<feed xmlns="http://www.w3.org/2005/Atom"><title>Andrew Ridgway's Blog - Resume</title><link href="http://localhost:8000/" rel="alternate"></link><link href="http://localhost:8000/feeds/resume.atom.xml" rel="self"></link><id>http://localhost:8000/</id><updated>2024-03-13T20:00:00+10:00</updated><entry><title>A Resume</title><link href="http://localhost:8000/cover-letter.html" rel="alternate"></link><published>2024-02-23T20:00:00+10:00</published><updated>2024-03-13T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2024-02-23:/cover-letter.html</id><summary type="html"><p>A Summary of My work Experience</p></summary><content type="html"><h1>OVERVIEW</h1>
|
||||
<p>I am a Senior Data Engineer looking to transition my skills to Data and Solution
|
||||
Architecting as well as project management. I have spent the better part of the
|
||||
last decade refining my abilities in taking business requirements and turning
|
||||
those into actionable data engineering, analytics, and software projects with
|
||||
trackable metrics. I believe in agnosticism when it comes to coding languages
|
||||
and have experimented in my own time with many different languages. In my
|
||||
career I have used Python, .NET, PowerShell, TSQL, VB and SAS (multiple
|
||||
products) in an Enterprise capacity. I also have experience using Google Cloud
|
||||
Platform and AWS tools for ETL and data platform development as well as git
|
||||
for version control and deployment using various IAC tools. I have also
|
||||
conducted data analysis and modelling on business metrics to find relationships
|
||||
between both staff and customer behavior and produced actionable
|
||||
recommendations based on the conclusions. In a private context I have also
|
||||
experimented with C, C# and Kotlin I am looking to further my career by taking
|
||||
my passion for data engineering and analysis as well as web and software
|
||||
development and applying it in a strategic context.</p>
|
||||
<h1>SKILLS &amp; ABILITIES</h1>
|
||||
<ul>
|
||||
<li>Python (scripting, compiling, notebooks – Sagemaker, Jupyter)</li>
|
||||
<li>git</li>
|
||||
<li>SAS (Base, EG, VA)</li>
|
||||
<li>Various Google Cloud Tools (Data Fusion, Compute Engine, Cloud Functions)</li>
|
||||
<li>Various Amazon Tools (EC2, RDS, Kinesis, Glue, Redshift, Lambda, ECS, ECR, EKS)</li>
|
||||
<li>Streaming Technologies (Kafka, Hive, Spark Streaming)</li>
|
||||
<li>Various DB platforms both on Prem and Serverless (MariaDB/MySql,</li>
|
||||
<li>Postgres/Redshift, SQL Server, RDS/Aurora variants)</li>
|
||||
<li>Various Microsoft Products (PowerBI, TSQL, Excel, VBA)</li>
|
||||
<li>Linux Server Administration (cron, bash, systemD)</li>
|
||||
<li>ETL/ELT Development</li>
|
||||
<li>Basic Data Modelling (Kimball, SCD Type 2)</li>
|
||||
<li>IAC (Cloud Formation, Terraform)</li>
|
||||
<li>Datahub Deployment</li>
|
||||
<li>Dagster Orchestration Deployments</li>
|
||||
<li>DBT Modelling and Design Deployments</li>
|
||||
<li>Containerised and Cloud Driven Data Architecture</li>
|
||||
</ul>
|
||||
<h1>EXPERIENCE</h1>
|
||||
<h2>Cloud Data Architect, Redeye Apps</h2>
|
||||
<p><em>May 2022 - Present</em>
|
||||
- Greenfields Research, Design and Deployment of S3 datalake (Parquet)
|
||||
- AWS DMS, S3, Athena, Glue
|
||||
- Research Design and Deployment of Catalog (Datahub)
|
||||
- Design of Data Governance Process (Datahub driven)
|
||||
- Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)
|
||||
- CI/CD design and deployment of modelling and orchestration using Gitlab
|
||||
- Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy
|
||||
- Design of ETL/Pipelines (DBT)
|
||||
- Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</p>
|
||||
<h2>Data Engineer, TechConnect IT Solutions</h2>
|
||||
<p><em>August 2021 – May 2022</em>
|
||||
- Design of Cloud Data Batch ETL solutions using Python (Glue)
|
||||
- Design of Cloud Data Streaming ETL solution using Python (Kinesis)
|
||||
- Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs
|
||||
- Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)
|
||||
- Apply Prebuilt ML models to software solutions (Sagemaker)
|
||||
- Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)
|
||||
- API testing and development (gRPC, Rest)</p>
|
||||
<h2>Enterprise Data Warehouse Developer, Auto and General Insurance</h2>
|
||||
<p><em>August 2019 - August 2021</em>
|
||||
- ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL
|
||||
- Bringing new data to the business to analyse for new insights
|
||||
- Redeveloped Version Control and brought git to the data team
|
||||
- Introduced python for API enablement in the Enterprise Data Warehouse
|
||||
- Partnering with the business to focus data project on actual need and translating into technical requirements</p>
|
||||
<h2>Business Analyst, Auto and General Insurance</h2>
|
||||
<p><em>January 2018 - August 2019</em>
|
||||
- Automate Service Performance Reporting using PowerShell/VBA/SAS
|
||||
- Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting
|
||||
- Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends
|
||||
- Technologies used include VBA, PowerShell, SQL, Web API’s, SAS
|
||||
- Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel
|
||||
- Gather Requirements to build meaningful reporting solutions
|
||||
- Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</p>
|
||||
<h2>Forecasting and Capacity Analyst, Auto and General Insurance</h2>
|
||||
<p><em>January 2017 – January 2018</em>
|
||||
- Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers
|
||||
- This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.
|
||||
- Forecast inbound workload requirements for the Auto and General sales call center using time series analysis
|
||||
- Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation
|
||||
- Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy
|
||||
- Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders
|
||||
- Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</p>
|
||||
<h2>Senior HR Performance Analyst, Queensland Department of Justice and Attorney General</h2>
|
||||
<p><em>June 2016 - January 2017</em>
|
||||
- Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics
|
||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
||||
<h2>Workforce Business Analyst, Queensland Department of Justice and Attorney General</h2>
|
||||
<p><em>July 2015 – June 2016</em>
|
||||
- Develop and refine current workforce analysis techniques and databases
|
||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||
- Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit
|
||||
- Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required
|
||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
||||
<h1>EDUCATION</h1>
|
||||
<ul>
|
||||
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
||||
<li>2008 Bachelor of Arts, University of Queensland</li>
|
||||
</ul>
|
||||
<h1>REFERENCES</h1>
|
||||
<ul>
|
||||
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health
|
||||
<em>0428 038 031</em></li>
|
||||
<li>Jaime Brian Head of Cloud Ninjas, TechConnect
|
||||
<em>0422 012 17</em></li>
|
||||
</ul></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry><entry><title>A Cover Letter</title><link href="http://localhost:8000/cover-letter.html" rel="alternate"></link><published>2024-02-23T20:00:00+10:00</published><updated>2024-03-13T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2024-02-23:/cover-letter.html</id><summary type="html"><p>A Summary of what I've done and Where I'd like to go for prospective Employers</p></summary><content type="html"><p>To whom it may concern</p>
|
||||
<p>My name is Andrew Ridgway and I am a Data and Technology professional looking to embark on the next step in my career.</p>
|
||||
<p>I have over 10 years’ experience in System and Data Architecture, Data Modelling and Orchestration, Business and Technical Analysis and System and Development Process Design. Most of this has been in developing Cloud architectures and workloads on AWS and GCP Including ML workloads using Sagemaker. </p>
|
||||
<p>In my current role I have Proposed, Designed and built the data platform currently used by business. This includes internal and external data products as well as the infrastructure and modelling to support these. This role has seen me liaise with stakeholders of all levels of the business from Analysts in the Customer Experience team right up to C suite executives and preparing material for board members. I understand the complexity of communicating complex system design to different level stakeholders and the complexities of involved in communicating to both technical and less technical employees particularly in relation to data and ML technologies. </p>
|
||||
|
||||
Loading…
x
Reference in New Issue
Block a user