fix formatting
This commit is contained in:
Andrew Ridgway
parent
40f425eb68
commit
e7794379fb
@ -45,8 +45,9 @@ development and applying it in a strategic context.
|
|||||||
- Containerised and Cloud Driven Data Architecture
|
- Containerised and Cloud Driven Data Architecture
|
||||||
|
|
||||||
# EXPERIENCE
|
# EXPERIENCE
|
||||||
## Cloud Data Architect, Redeye Apps
|
## Cloud Data Architect
|
||||||
_May 2022 - Present_
|
### _Redeye Apps_
|
||||||
|
#### _May 2022 - Present_
|
||||||
- Greenfields Research, Design and Deployment of S3 datalake (Parquet)
|
- Greenfields Research, Design and Deployment of S3 datalake (Parquet)
|
||||||
- AWS DMS, S3, Athena, Glue
|
- AWS DMS, S3, Athena, Glue
|
||||||
- Research Design and Deployment of Catalog (Datahub)
|
- Research Design and Deployment of Catalog (Datahub)
|
||||||
@ -56,8 +57,10 @@ _May 2022 - Present_
|
|||||||
- Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy
|
- Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy
|
||||||
- Design of ETL/Pipelines (DBT)
|
- Design of ETL/Pipelines (DBT)
|
||||||
- Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)
|
- Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)
|
||||||
## Data Engineer, TechConnect IT Solutions
|
|
||||||
_August 2021 – May 2022_
|
## Data Engineer,
|
||||||
|
### _TechConnect IT Solutions_
|
||||||
|
#### _August 2021 – May 2022_
|
||||||
- Design of Cloud Data Batch ETL solutions using Python (Glue)
|
- Design of Cloud Data Batch ETL solutions using Python (Glue)
|
||||||
- Design of Cloud Data Streaming ETL solution using Python (Kinesis)
|
- Design of Cloud Data Streaming ETL solution using Python (Kinesis)
|
||||||
- Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs
|
- Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs
|
||||||
@ -65,16 +68,19 @@ _August 2021 – May 2022_
|
|||||||
- Apply Prebuilt ML models to software solutions (Sagemaker)
|
- Apply Prebuilt ML models to software solutions (Sagemaker)
|
||||||
- Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)
|
- Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)
|
||||||
- API testing and development (gRPC, Rest)
|
- API testing and development (gRPC, Rest)
|
||||||
## Enterprise Data Warehouse Developer, Auto and General Insurance
|
|
||||||
_August 2019 - August 2021_
|
## Enterprise Data Warehouse Developer
|
||||||
|
### _Auto and General Insurance_
|
||||||
|
#### _August 2019 - August 2021_
|
||||||
- ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL
|
- ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL
|
||||||
- Bringing new data to the business to analyse for new insights
|
- Bringing new data to the business to analyse for new insights
|
||||||
- Redeveloped Version Control and brought git to the data team
|
- Redeveloped Version Control and brought git to the data team
|
||||||
- Introduced python for API enablement in the Enterprise Data Warehouse
|
- Introduced python for API enablement in the Enterprise Data Warehouse
|
||||||
- Partnering with the business to focus data project on actual need and translating into technical requirements
|
- Partnering with the business to focus data project on actual need and translating into technical requirements
|
||||||
|
|
||||||
## Business Analyst, Auto and General Insurance
|
## Business Analyst
|
||||||
_January 2018 - August 2019_
|
### _Auto and General Insurance_
|
||||||
|
#### _January 2018 - August 2019_
|
||||||
- Automate Service Performance Reporting using PowerShell/VBA/SAS
|
- Automate Service Performance Reporting using PowerShell/VBA/SAS
|
||||||
- Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting
|
- Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting
|
||||||
- Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends
|
- Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends
|
||||||
@ -83,8 +89,9 @@ _January 2018 - August 2019_
|
|||||||
- Gather Requirements to build meaningful reporting solutions
|
- Gather Requirements to build meaningful reporting solutions
|
||||||
- Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.
|
- Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.
|
||||||
|
|
||||||
## Forecasting and Capacity Analyst, Auto and General Insurance
|
## Forecasting and Capacity Analyst
|
||||||
_January 2017 – January 2018_
|
### _Auto and General Insurance_
|
||||||
|
#### _January 2017 – January 2018_
|
||||||
- Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers
|
- Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers
|
||||||
- This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.
|
- This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.
|
||||||
- Forecast inbound workload requirements for the Auto and General sales call center using time series analysis
|
- Forecast inbound workload requirements for the Auto and General sales call center using time series analysis
|
||||||
@ -93,14 +100,16 @@ _January 2017 – January 2018_
|
|||||||
- Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders
|
- Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders
|
||||||
- Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave
|
- Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave
|
||||||
|
|
||||||
## Senior HR Performance Analyst, Queensland Department of Justice and Attorney General
|
## Senior HR Performance Analyst
|
||||||
_June 2016 - January 2017_
|
### _Queensland Department of Justice and Attorney General_
|
||||||
|
#### _June 2016 - January 2017_
|
||||||
- Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics
|
- Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics
|
||||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives
|
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives
|
||||||
|
|
||||||
## Workforce Business Analyst, Queensland Department of Justice and Attorney General
|
## Workforce Business Analyst
|
||||||
_July 2015 – June 2016_
|
### _Queensland Department of Justice and Attorney General_
|
||||||
|
#### _July 2015 – June 2016_
|
||||||
- Develop and refine current workforce analysis techniques and databases
|
- Develop and refine current workforce analysis techniques and databases
|
||||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
||||||
- Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit
|
- Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit
|
||||||
@ -113,7 +122,10 @@ _July 2015 – June 2016_
|
|||||||
|
|
||||||
# REFERENCES
|
# REFERENCES
|
||||||
- Anthony Stiller Lead Developer, Data warehousing, Queensland Health
|
- Anthony Stiller Lead Developer, Data warehousing, Queensland Health
|
||||||
|
|
||||||
_0428 038 031_
|
_0428 038 031_
|
||||||
|
|
||||||
- Jaime Brian Head of Cloud Ninjas, TechConnect
|
- Jaime Brian Head of Cloud Ninjas, TechConnect
|
||||||
|
|
||||||
_0422 012 17_
|
_0422 012 17_
|
||||||
|
|
||||||
|
|||||||
@ -49,63 +49,84 @@ development and applying it in a strategic context.</p>
|
|||||||
<li>Containerised and Cloud Driven Data Architecture</li>
|
<li>Containerised and Cloud Driven Data Architecture</li>
|
||||||
</ul>
|
</ul>
|
||||||
<h1>EXPERIENCE</h1>
|
<h1>EXPERIENCE</h1>
|
||||||
<h2>Cloud Data Architect, Redeye Apps</h2>
|
<h2>Cloud Data Architect</h2>
|
||||||
<p><em>May 2022 - Present</em>
|
<h3><em>Redeye Apps</em></h3>
|
||||||
- Greenfields Research, Design and Deployment of S3 datalake (Parquet)
|
<h4><em>May 2022 - Present</em></h4>
|
||||||
- AWS DMS, S3, Athena, Glue
|
<ul>
|
||||||
- Research Design and Deployment of Catalog (Datahub)
|
<li>Greenfields Research, Design and Deployment of S3 datalake (Parquet)</li>
|
||||||
- Design of Data Governance Process (Datahub driven)
|
<li>AWS DMS, S3, Athena, Glue</li>
|
||||||
- Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)
|
<li>Research Design and Deployment of Catalog (Datahub)</li>
|
||||||
- CI/CD design and deployment of modelling and orchestration using Gitlab
|
<li>Design of Data Governance Process (Datahub driven)</li>
|
||||||
- Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy
|
<li>Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)</li>
|
||||||
- Design of ETL/Pipelines (DBT)
|
<li>CI/CD design and deployment of modelling and orchestration using Gitlab</li>
|
||||||
- Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</p>
|
<li>Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy</li>
|
||||||
<h2>Data Engineer, TechConnect IT Solutions</h2>
|
<li>Design of ETL/Pipelines (DBT)</li>
|
||||||
<p><em>August 2021 – May 2022</em>
|
<li>Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</li>
|
||||||
- Design of Cloud Data Batch ETL solutions using Python (Glue)
|
</ul>
|
||||||
- Design of Cloud Data Streaming ETL solution using Python (Kinesis)
|
<h2>Data Engineer,</h2>
|
||||||
- Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs
|
<h3><em>TechConnect IT Solutions</em></h3>
|
||||||
- Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)
|
<h4><em>August 2021 – May 2022</em></h4>
|
||||||
- Apply Prebuilt ML models to software solutions (Sagemaker)
|
<ul>
|
||||||
- Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)
|
<li>Design of Cloud Data Batch ETL solutions using Python (Glue)</li>
|
||||||
- API testing and development (gRPC, Rest)</p>
|
<li>Design of Cloud Data Streaming ETL solution using Python (Kinesis)</li>
|
||||||
<h2>Enterprise Data Warehouse Developer, Auto and General Insurance</h2>
|
<li>Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs</li>
|
||||||
<p><em>August 2019 - August 2021</em>
|
<li>Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)</li>
|
||||||
- ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL
|
<li>Apply Prebuilt ML models to software solutions (Sagemaker)</li>
|
||||||
- Bringing new data to the business to analyse for new insights
|
<li>Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)</li>
|
||||||
- Redeveloped Version Control and brought git to the data team
|
<li>API testing and development (gRPC, Rest)</li>
|
||||||
- Introduced python for API enablement in the Enterprise Data Warehouse
|
</ul>
|
||||||
- Partnering with the business to focus data project on actual need and translating into technical requirements</p>
|
<h2>Enterprise Data Warehouse Developer</h2>
|
||||||
<h2>Business Analyst, Auto and General Insurance</h2>
|
<h3><em>Auto and General Insurance</em></h3>
|
||||||
<p><em>January 2018 - August 2019</em>
|
<h4><em>August 2019 - August 2021</em></h4>
|
||||||
- Automate Service Performance Reporting using PowerShell/VBA/SAS
|
<ul>
|
||||||
- Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting
|
<li>ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL</li>
|
||||||
- Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends
|
<li>Bringing new data to the business to analyse for new insights</li>
|
||||||
- Technologies used include VBA, PowerShell, SQL, Web API’s, SAS
|
<li>Redeveloped Version Control and brought git to the data team</li>
|
||||||
- Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel
|
<li>Introduced python for API enablement in the Enterprise Data Warehouse</li>
|
||||||
- Gather Requirements to build meaningful reporting solutions
|
<li>Partnering with the business to focus data project on actual need and translating into technical requirements</li>
|
||||||
- Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</p>
|
</ul>
|
||||||
<h2>Forecasting and Capacity Analyst, Auto and General Insurance</h2>
|
<h2>Business Analyst</h2>
|
||||||
<p><em>January 2017 – January 2018</em>
|
<h3><em>Auto and General Insurance</em></h3>
|
||||||
- Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers
|
<h4><em>January 2018 - August 2019</em></h4>
|
||||||
- This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.
|
<ul>
|
||||||
- Forecast inbound workload requirements for the Auto and General sales call center using time series analysis
|
<li>Automate Service Performance Reporting using PowerShell/VBA/SAS</li>
|
||||||
- Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation
|
<li>Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting</li>
|
||||||
- Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy
|
<li>Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends</li>
|
||||||
- Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders
|
<li>Technologies used include VBA, PowerShell, SQL, Web API’s, SAS</li>
|
||||||
- Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</p>
|
<li>Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel</li>
|
||||||
<h2>Senior HR Performance Analyst, Queensland Department of Justice and Attorney General</h2>
|
<li>Gather Requirements to build meaningful reporting solutions</li>
|
||||||
<p><em>June 2016 - January 2017</em>
|
<li>Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</li>
|
||||||
- Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics
|
</ul>
|
||||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
<h2>Forecasting and Capacity Analyst</h2>
|
||||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
<h3><em>Auto and General Insurance</em></h3>
|
||||||
<h2>Workforce Business Analyst, Queensland Department of Justice and Attorney General</h2>
|
<h4><em>January 2017 – January 2018</em></h4>
|
||||||
<p><em>July 2015 – June 2016</em>
|
<ul>
|
||||||
- Develop and refine current workforce analysis techniques and databases
|
<li>Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers</li>
|
||||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
<li>This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.</li>
|
||||||
- Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit
|
<li>Forecast inbound workload requirements for the Auto and General sales call center using time series analysis</li>
|
||||||
- Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required
|
<li>Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation</li>
|
||||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
<li>Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy</li>
|
||||||
|
<li>Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders</li>
|
||||||
|
<li>Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</li>
|
||||||
|
</ul>
|
||||||
|
<h2>Senior HR Performance Analyst</h2>
|
||||||
|
<h3><em>Queensland Department of Justice and Attorney General</em></h3>
|
||||||
|
<h4><em>June 2016 - January 2017</em></h4>
|
||||||
|
<ul>
|
||||||
|
<li>Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics</li>
|
||||||
|
<li>Use VBA to automate regular reporting in Microsoft Access and Excel</li>
|
||||||
|
<li>Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</li>
|
||||||
|
</ul>
|
||||||
|
<h2>Workforce Business Analyst</h2>
|
||||||
|
<h3><em>Queensland Department of Justice and Attorney General</em></h3>
|
||||||
|
<h4><em>July 2015 – June 2016</em></h4>
|
||||||
|
<ul>
|
||||||
|
<li>Develop and refine current workforce analysis techniques and databases</li>
|
||||||
|
<li>Use VBA to automate regular reporting in Microsoft Access and Excel</li>
|
||||||
|
<li>Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit</li>
|
||||||
|
<li>Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required</li>
|
||||||
|
<li>Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</li>
|
||||||
|
</ul>
|
||||||
<h1>EDUCATION</h1>
|
<h1>EDUCATION</h1>
|
||||||
<ul>
|
<ul>
|
||||||
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
||||||
@ -113,11 +134,13 @@ development and applying it in a strategic context.</p>
|
|||||||
</ul>
|
</ul>
|
||||||
<h1>REFERENCES</h1>
|
<h1>REFERENCES</h1>
|
||||||
<ul>
|
<ul>
|
||||||
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health
|
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health</li>
|
||||||
<em>0428 038 031</em></li>
|
</ul>
|
||||||
<li>Jaime Brian Head of Cloud Ninjas, TechConnect
|
<p><em>0428 038 031</em></p>
|
||||||
<em>0422 012 17</em></li>
|
<ul>
|
||||||
</ul></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry><entry><title>Metabase and DuckDB</title><link href="http://localhost:8000/metabase-duckdb.html" rel="alternate"></link><published>2023-11-15T20:00:00+10:00</published><updated>2023-11-15T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2023-11-15:/metabase-duckdb.html</id><summary type="html"><p>Using Metabase and DuckDB to create an embedded Reporting Container bringing the data as close to the report as possible</p></summary><content type="html"><p>Ahhhh <a href="https://duckdb.org/">DuckDB</a> if you're even partly floating around in the data space you've probably been hearing ALOT about it and it's <em>"Datawarehouse on your laptop"</em> mantra. However, the OTHER application that sometimes gets missed is <em>"SQLite for OLAP workloads"</em> and it was this concept that once I grasped it gave me a very interesting idea.... What if we could take the very pretty Aggregate Layer of our Data(warehouse/LakeHouse/Lake) and put that data right next to presentation layer of the lake, reducing network latency and... hopefully... have presentation reports running over very large workloads in the blink of an eye. It might even be fast enough that it could be deployed and embedded </p>
|
<li>Jaime Brian Head of Cloud Ninjas, TechConnect</li>
|
||||||
|
</ul>
|
||||||
|
<p><em>0422 012 17</em></p></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry><entry><title>Metabase and DuckDB</title><link href="http://localhost:8000/metabase-duckdb.html" rel="alternate"></link><published>2023-11-15T20:00:00+10:00</published><updated>2023-11-15T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2023-11-15:/metabase-duckdb.html</id><summary type="html"><p>Using Metabase and DuckDB to create an embedded Reporting Container bringing the data as close to the report as possible</p></summary><content type="html"><p>Ahhhh <a href="https://duckdb.org/">DuckDB</a> if you're even partly floating around in the data space you've probably been hearing ALOT about it and it's <em>"Datawarehouse on your laptop"</em> mantra. However, the OTHER application that sometimes gets missed is <em>"SQLite for OLAP workloads"</em> and it was this concept that once I grasped it gave me a very interesting idea.... What if we could take the very pretty Aggregate Layer of our Data(warehouse/LakeHouse/Lake) and put that data right next to presentation layer of the lake, reducing network latency and... hopefully... have presentation reports running over very large workloads in the blink of an eye. It might even be fast enough that it could be deployed and embedded </p>
|
||||||
<p>However, for this to work we need some form of conatinerised reporting application.... lucky for us there is <a href="https://www.metabase.com/">Metabase</a> which is a fantastic little reporting application that has an open core. So this got me thinking... Can I put these two applications together and create a Reporting Layer with report embedding capabilities that is deployable in the cluster and has a admin UI accesible over a web page all whilst keeping the data locked to our network?</p>
|
<p>However, for this to work we need some form of conatinerised reporting application.... lucky for us there is <a href="https://www.metabase.com/">Metabase</a> which is a fantastic little reporting application that has an open core. So this got me thinking... Can I put these two applications together and create a Reporting Layer with report embedding capabilities that is deployable in the cluster and has a admin UI accesible over a web page all whilst keeping the data locked to our network?</p>
|
||||||
<h3>The Beginnings of an Idea</h3>
|
<h3>The Beginnings of an Idea</h3>
|
||||||
<p>Ok so... Big first question. Can Duckdb and Metabase talk? Well... not quite. But first lets take a quick look at the architecture we'll be employing here </p>
|
<p>Ok so... Big first question. Can Duckdb and Metabase talk? Well... not quite. But first lets take a quick look at the architecture we'll be employing here </p>
|
||||||
|
|||||||
@ -49,63 +49,84 @@ development and applying it in a strategic context.</p>
|
|||||||
<li>Containerised and Cloud Driven Data Architecture</li>
|
<li>Containerised and Cloud Driven Data Architecture</li>
|
||||||
</ul>
|
</ul>
|
||||||
<h1>EXPERIENCE</h1>
|
<h1>EXPERIENCE</h1>
|
||||||
<h2>Cloud Data Architect, Redeye Apps</h2>
|
<h2>Cloud Data Architect</h2>
|
||||||
<p><em>May 2022 - Present</em>
|
<h3><em>Redeye Apps</em></h3>
|
||||||
- Greenfields Research, Design and Deployment of S3 datalake (Parquet)
|
<h4><em>May 2022 - Present</em></h4>
|
||||||
- AWS DMS, S3, Athena, Glue
|
<ul>
|
||||||
- Research Design and Deployment of Catalog (Datahub)
|
<li>Greenfields Research, Design and Deployment of S3 datalake (Parquet)</li>
|
||||||
- Design of Data Governance Process (Datahub driven)
|
<li>AWS DMS, S3, Athena, Glue</li>
|
||||||
- Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)
|
<li>Research Design and Deployment of Catalog (Datahub)</li>
|
||||||
- CI/CD design and deployment of modelling and orchestration using Gitlab
|
<li>Design of Data Governance Process (Datahub driven)</li>
|
||||||
- Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy
|
<li>Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)</li>
|
||||||
- Design of ETL/Pipelines (DBT)
|
<li>CI/CD design and deployment of modelling and orchestration using Gitlab</li>
|
||||||
- Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</p>
|
<li>Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy</li>
|
||||||
<h2>Data Engineer, TechConnect IT Solutions</h2>
|
<li>Design of ETL/Pipelines (DBT)</li>
|
||||||
<p><em>August 2021 – May 2022</em>
|
<li>Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</li>
|
||||||
- Design of Cloud Data Batch ETL solutions using Python (Glue)
|
</ul>
|
||||||
- Design of Cloud Data Streaming ETL solution using Python (Kinesis)
|
<h2>Data Engineer,</h2>
|
||||||
- Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs
|
<h3><em>TechConnect IT Solutions</em></h3>
|
||||||
- Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)
|
<h4><em>August 2021 – May 2022</em></h4>
|
||||||
- Apply Prebuilt ML models to software solutions (Sagemaker)
|
<ul>
|
||||||
- Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)
|
<li>Design of Cloud Data Batch ETL solutions using Python (Glue)</li>
|
||||||
- API testing and development (gRPC, Rest)</p>
|
<li>Design of Cloud Data Streaming ETL solution using Python (Kinesis)</li>
|
||||||
<h2>Enterprise Data Warehouse Developer, Auto and General Insurance</h2>
|
<li>Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs</li>
|
||||||
<p><em>August 2019 - August 2021</em>
|
<li>Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)</li>
|
||||||
- ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL
|
<li>Apply Prebuilt ML models to software solutions (Sagemaker)</li>
|
||||||
- Bringing new data to the business to analyse for new insights
|
<li>Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)</li>
|
||||||
- Redeveloped Version Control and brought git to the data team
|
<li>API testing and development (gRPC, Rest)</li>
|
||||||
- Introduced python for API enablement in the Enterprise Data Warehouse
|
</ul>
|
||||||
- Partnering with the business to focus data project on actual need and translating into technical requirements</p>
|
<h2>Enterprise Data Warehouse Developer</h2>
|
||||||
<h2>Business Analyst, Auto and General Insurance</h2>
|
<h3><em>Auto and General Insurance</em></h3>
|
||||||
<p><em>January 2018 - August 2019</em>
|
<h4><em>August 2019 - August 2021</em></h4>
|
||||||
- Automate Service Performance Reporting using PowerShell/VBA/SAS
|
<ul>
|
||||||
- Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting
|
<li>ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL</li>
|
||||||
- Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends
|
<li>Bringing new data to the business to analyse for new insights</li>
|
||||||
- Technologies used include VBA, PowerShell, SQL, Web API’s, SAS
|
<li>Redeveloped Version Control and brought git to the data team</li>
|
||||||
- Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel
|
<li>Introduced python for API enablement in the Enterprise Data Warehouse</li>
|
||||||
- Gather Requirements to build meaningful reporting solutions
|
<li>Partnering with the business to focus data project on actual need and translating into technical requirements</li>
|
||||||
- Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</p>
|
</ul>
|
||||||
<h2>Forecasting and Capacity Analyst, Auto and General Insurance</h2>
|
<h2>Business Analyst</h2>
|
||||||
<p><em>January 2017 – January 2018</em>
|
<h3><em>Auto and General Insurance</em></h3>
|
||||||
- Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers
|
<h4><em>January 2018 - August 2019</em></h4>
|
||||||
- This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.
|
<ul>
|
||||||
- Forecast inbound workload requirements for the Auto and General sales call center using time series analysis
|
<li>Automate Service Performance Reporting using PowerShell/VBA/SAS</li>
|
||||||
- Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation
|
<li>Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting</li>
|
||||||
- Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy
|
<li>Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends</li>
|
||||||
- Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders
|
<li>Technologies used include VBA, PowerShell, SQL, Web API’s, SAS</li>
|
||||||
- Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</p>
|
<li>Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel</li>
|
||||||
<h2>Senior HR Performance Analyst, Queensland Department of Justice and Attorney General</h2>
|
<li>Gather Requirements to build meaningful reporting solutions</li>
|
||||||
<p><em>June 2016 - January 2017</em>
|
<li>Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</li>
|
||||||
- Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics
|
</ul>
|
||||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
<h2>Forecasting and Capacity Analyst</h2>
|
||||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
<h3><em>Auto and General Insurance</em></h3>
|
||||||
<h2>Workforce Business Analyst, Queensland Department of Justice and Attorney General</h2>
|
<h4><em>January 2017 – January 2018</em></h4>
|
||||||
<p><em>July 2015 – June 2016</em>
|
<ul>
|
||||||
- Develop and refine current workforce analysis techniques and databases
|
<li>Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers</li>
|
||||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
<li>This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.</li>
|
||||||
- Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit
|
<li>Forecast inbound workload requirements for the Auto and General sales call center using time series analysis</li>
|
||||||
- Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required
|
<li>Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation</li>
|
||||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
<li>Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy</li>
|
||||||
|
<li>Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders</li>
|
||||||
|
<li>Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</li>
|
||||||
|
</ul>
|
||||||
|
<h2>Senior HR Performance Analyst</h2>
|
||||||
|
<h3><em>Queensland Department of Justice and Attorney General</em></h3>
|
||||||
|
<h4><em>June 2016 - January 2017</em></h4>
|
||||||
|
<ul>
|
||||||
|
<li>Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics</li>
|
||||||
|
<li>Use VBA to automate regular reporting in Microsoft Access and Excel</li>
|
||||||
|
<li>Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</li>
|
||||||
|
</ul>
|
||||||
|
<h2>Workforce Business Analyst</h2>
|
||||||
|
<h3><em>Queensland Department of Justice and Attorney General</em></h3>
|
||||||
|
<h4><em>July 2015 – June 2016</em></h4>
|
||||||
|
<ul>
|
||||||
|
<li>Develop and refine current workforce analysis techniques and databases</li>
|
||||||
|
<li>Use VBA to automate regular reporting in Microsoft Access and Excel</li>
|
||||||
|
<li>Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit</li>
|
||||||
|
<li>Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required</li>
|
||||||
|
<li>Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</li>
|
||||||
|
</ul>
|
||||||
<h1>EDUCATION</h1>
|
<h1>EDUCATION</h1>
|
||||||
<ul>
|
<ul>
|
||||||
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
||||||
@ -113,11 +134,13 @@ development and applying it in a strategic context.</p>
|
|||||||
</ul>
|
</ul>
|
||||||
<h1>REFERENCES</h1>
|
<h1>REFERENCES</h1>
|
||||||
<ul>
|
<ul>
|
||||||
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health
|
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health</li>
|
||||||
<em>0428 038 031</em></li>
|
</ul>
|
||||||
<li>Jaime Brian Head of Cloud Ninjas, TechConnect
|
<p><em>0428 038 031</em></p>
|
||||||
<em>0422 012 17</em></li>
|
<ul>
|
||||||
</ul></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry><entry><title>Metabase and DuckDB</title><link href="http://localhost:8000/metabase-duckdb.html" rel="alternate"></link><published>2023-11-15T20:00:00+10:00</published><updated>2023-11-15T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2023-11-15:/metabase-duckdb.html</id><summary type="html"><p>Using Metabase and DuckDB to create an embedded Reporting Container bringing the data as close to the report as possible</p></summary><content type="html"><p>Ahhhh <a href="https://duckdb.org/">DuckDB</a> if you're even partly floating around in the data space you've probably been hearing ALOT about it and it's <em>"Datawarehouse on your laptop"</em> mantra. However, the OTHER application that sometimes gets missed is <em>"SQLite for OLAP workloads"</em> and it was this concept that once I grasped it gave me a very interesting idea.... What if we could take the very pretty Aggregate Layer of our Data(warehouse/LakeHouse/Lake) and put that data right next to presentation layer of the lake, reducing network latency and... hopefully... have presentation reports running over very large workloads in the blink of an eye. It might even be fast enough that it could be deployed and embedded </p>
|
<li>Jaime Brian Head of Cloud Ninjas, TechConnect</li>
|
||||||
|
</ul>
|
||||||
|
<p><em>0422 012 17</em></p></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry><entry><title>Metabase and DuckDB</title><link href="http://localhost:8000/metabase-duckdb.html" rel="alternate"></link><published>2023-11-15T20:00:00+10:00</published><updated>2023-11-15T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2023-11-15:/metabase-duckdb.html</id><summary type="html"><p>Using Metabase and DuckDB to create an embedded Reporting Container bringing the data as close to the report as possible</p></summary><content type="html"><p>Ahhhh <a href="https://duckdb.org/">DuckDB</a> if you're even partly floating around in the data space you've probably been hearing ALOT about it and it's <em>"Datawarehouse on your laptop"</em> mantra. However, the OTHER application that sometimes gets missed is <em>"SQLite for OLAP workloads"</em> and it was this concept that once I grasped it gave me a very interesting idea.... What if we could take the very pretty Aggregate Layer of our Data(warehouse/LakeHouse/Lake) and put that data right next to presentation layer of the lake, reducing network latency and... hopefully... have presentation reports running over very large workloads in the blink of an eye. It might even be fast enough that it could be deployed and embedded </p>
|
||||||
<p>However, for this to work we need some form of conatinerised reporting application.... lucky for us there is <a href="https://www.metabase.com/">Metabase</a> which is a fantastic little reporting application that has an open core. So this got me thinking... Can I put these two applications together and create a Reporting Layer with report embedding capabilities that is deployable in the cluster and has a admin UI accesible over a web page all whilst keeping the data locked to our network?</p>
|
<p>However, for this to work we need some form of conatinerised reporting application.... lucky for us there is <a href="https://www.metabase.com/">Metabase</a> which is a fantastic little reporting application that has an open core. So this got me thinking... Can I put these two applications together and create a Reporting Layer with report embedding capabilities that is deployable in the cluster and has a admin UI accesible over a web page all whilst keeping the data locked to our network?</p>
|
||||||
<h3>The Beginnings of an Idea</h3>
|
<h3>The Beginnings of an Idea</h3>
|
||||||
<p>Ok so... Big first question. Can Duckdb and Metabase talk? Well... not quite. But first lets take a quick look at the architecture we'll be employing here </p>
|
<p>Ok so... Big first question. Can Duckdb and Metabase talk? Well... not quite. But first lets take a quick look at the architecture we'll be employing here </p>
|
||||||
|
|||||||
@ -49,63 +49,84 @@ development and applying it in a strategic context.</p>
|
|||||||
<li>Containerised and Cloud Driven Data Architecture</li>
|
<li>Containerised and Cloud Driven Data Architecture</li>
|
||||||
</ul>
|
</ul>
|
||||||
<h1>EXPERIENCE</h1>
|
<h1>EXPERIENCE</h1>
|
||||||
<h2>Cloud Data Architect, Redeye Apps</h2>
|
<h2>Cloud Data Architect</h2>
|
||||||
<p><em>May 2022 - Present</em>
|
<h3><em>Redeye Apps</em></h3>
|
||||||
- Greenfields Research, Design and Deployment of S3 datalake (Parquet)
|
<h4><em>May 2022 - Present</em></h4>
|
||||||
- AWS DMS, S3, Athena, Glue
|
<ul>
|
||||||
- Research Design and Deployment of Catalog (Datahub)
|
<li>Greenfields Research, Design and Deployment of S3 datalake (Parquet)</li>
|
||||||
- Design of Data Governance Process (Datahub driven)
|
<li>AWS DMS, S3, Athena, Glue</li>
|
||||||
- Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)
|
<li>Research Design and Deployment of Catalog (Datahub)</li>
|
||||||
- CI/CD design and deployment of modelling and orchestration using Gitlab
|
<li>Design of Data Governance Process (Datahub driven)</li>
|
||||||
- Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy
|
<li>Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)</li>
|
||||||
- Design of ETL/Pipelines (DBT)
|
<li>CI/CD design and deployment of modelling and orchestration using Gitlab</li>
|
||||||
- Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</p>
|
<li>Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy</li>
|
||||||
<h2>Data Engineer, TechConnect IT Solutions</h2>
|
<li>Design of ETL/Pipelines (DBT)</li>
|
||||||
<p><em>August 2021 – May 2022</em>
|
<li>Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</li>
|
||||||
- Design of Cloud Data Batch ETL solutions using Python (Glue)
|
</ul>
|
||||||
- Design of Cloud Data Streaming ETL solution using Python (Kinesis)
|
<h2>Data Engineer,</h2>
|
||||||
- Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs
|
<h3><em>TechConnect IT Solutions</em></h3>
|
||||||
- Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)
|
<h4><em>August 2021 – May 2022</em></h4>
|
||||||
- Apply Prebuilt ML models to software solutions (Sagemaker)
|
<ul>
|
||||||
- Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)
|
<li>Design of Cloud Data Batch ETL solutions using Python (Glue)</li>
|
||||||
- API testing and development (gRPC, Rest)</p>
|
<li>Design of Cloud Data Streaming ETL solution using Python (Kinesis)</li>
|
||||||
<h2>Enterprise Data Warehouse Developer, Auto and General Insurance</h2>
|
<li>Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs</li>
|
||||||
<p><em>August 2019 - August 2021</em>
|
<li>Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)</li>
|
||||||
- ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL
|
<li>Apply Prebuilt ML models to software solutions (Sagemaker)</li>
|
||||||
- Bringing new data to the business to analyse for new insights
|
<li>Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)</li>
|
||||||
- Redeveloped Version Control and brought git to the data team
|
<li>API testing and development (gRPC, Rest)</li>
|
||||||
- Introduced python for API enablement in the Enterprise Data Warehouse
|
</ul>
|
||||||
- Partnering with the business to focus data project on actual need and translating into technical requirements</p>
|
<h2>Enterprise Data Warehouse Developer</h2>
|
||||||
<h2>Business Analyst, Auto and General Insurance</h2>
|
<h3><em>Auto and General Insurance</em></h3>
|
||||||
<p><em>January 2018 - August 2019</em>
|
<h4><em>August 2019 - August 2021</em></h4>
|
||||||
- Automate Service Performance Reporting using PowerShell/VBA/SAS
|
<ul>
|
||||||
- Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting
|
<li>ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL</li>
|
||||||
- Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends
|
<li>Bringing new data to the business to analyse for new insights</li>
|
||||||
- Technologies used include VBA, PowerShell, SQL, Web API’s, SAS
|
<li>Redeveloped Version Control and brought git to the data team</li>
|
||||||
- Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel
|
<li>Introduced python for API enablement in the Enterprise Data Warehouse</li>
|
||||||
- Gather Requirements to build meaningful reporting solutions
|
<li>Partnering with the business to focus data project on actual need and translating into technical requirements</li>
|
||||||
- Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</p>
|
</ul>
|
||||||
<h2>Forecasting and Capacity Analyst, Auto and General Insurance</h2>
|
<h2>Business Analyst</h2>
|
||||||
<p><em>January 2017 – January 2018</em>
|
<h3><em>Auto and General Insurance</em></h3>
|
||||||
- Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers
|
<h4><em>January 2018 - August 2019</em></h4>
|
||||||
- This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.
|
<ul>
|
||||||
- Forecast inbound workload requirements for the Auto and General sales call center using time series analysis
|
<li>Automate Service Performance Reporting using PowerShell/VBA/SAS</li>
|
||||||
- Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation
|
<li>Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting</li>
|
||||||
- Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy
|
<li>Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends</li>
|
||||||
- Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders
|
<li>Technologies used include VBA, PowerShell, SQL, Web API’s, SAS</li>
|
||||||
- Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</p>
|
<li>Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel</li>
|
||||||
<h2>Senior HR Performance Analyst, Queensland Department of Justice and Attorney General</h2>
|
<li>Gather Requirements to build meaningful reporting solutions</li>
|
||||||
<p><em>June 2016 - January 2017</em>
|
<li>Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</li>
|
||||||
- Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics
|
</ul>
|
||||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
<h2>Forecasting and Capacity Analyst</h2>
|
||||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
<h3><em>Auto and General Insurance</em></h3>
|
||||||
<h2>Workforce Business Analyst, Queensland Department of Justice and Attorney General</h2>
|
<h4><em>January 2017 – January 2018</em></h4>
|
||||||
<p><em>July 2015 – June 2016</em>
|
<ul>
|
||||||
- Develop and refine current workforce analysis techniques and databases
|
<li>Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers</li>
|
||||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
<li>This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.</li>
|
||||||
- Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit
|
<li>Forecast inbound workload requirements for the Auto and General sales call center using time series analysis</li>
|
||||||
- Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required
|
<li>Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation</li>
|
||||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
<li>Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy</li>
|
||||||
|
<li>Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders</li>
|
||||||
|
<li>Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</li>
|
||||||
|
</ul>
|
||||||
|
<h2>Senior HR Performance Analyst</h2>
|
||||||
|
<h3><em>Queensland Department of Justice and Attorney General</em></h3>
|
||||||
|
<h4><em>June 2016 - January 2017</em></h4>
|
||||||
|
<ul>
|
||||||
|
<li>Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics</li>
|
||||||
|
<li>Use VBA to automate regular reporting in Microsoft Access and Excel</li>
|
||||||
|
<li>Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</li>
|
||||||
|
</ul>
|
||||||
|
<h2>Workforce Business Analyst</h2>
|
||||||
|
<h3><em>Queensland Department of Justice and Attorney General</em></h3>
|
||||||
|
<h4><em>July 2015 – June 2016</em></h4>
|
||||||
|
<ul>
|
||||||
|
<li>Develop and refine current workforce analysis techniques and databases</li>
|
||||||
|
<li>Use VBA to automate regular reporting in Microsoft Access and Excel</li>
|
||||||
|
<li>Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit</li>
|
||||||
|
<li>Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required</li>
|
||||||
|
<li>Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</li>
|
||||||
|
</ul>
|
||||||
<h1>EDUCATION</h1>
|
<h1>EDUCATION</h1>
|
||||||
<ul>
|
<ul>
|
||||||
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
||||||
@ -113,11 +134,13 @@ development and applying it in a strategic context.</p>
|
|||||||
</ul>
|
</ul>
|
||||||
<h1>REFERENCES</h1>
|
<h1>REFERENCES</h1>
|
||||||
<ul>
|
<ul>
|
||||||
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health
|
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health</li>
|
||||||
<em>0428 038 031</em></li>
|
</ul>
|
||||||
<li>Jaime Brian Head of Cloud Ninjas, TechConnect
|
<p><em>0428 038 031</em></p>
|
||||||
<em>0422 012 17</em></li>
|
<ul>
|
||||||
</ul></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry><entry><title>Metabase and DuckDB</title><link href="http://localhost:8000/metabase-duckdb.html" rel="alternate"></link><published>2023-11-15T20:00:00+10:00</published><updated>2023-11-15T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2023-11-15:/metabase-duckdb.html</id><summary type="html"><p>Using Metabase and DuckDB to create an embedded Reporting Container bringing the data as close to the report as possible</p></summary><content type="html"><p>Ahhhh <a href="https://duckdb.org/">DuckDB</a> if you're even partly floating around in the data space you've probably been hearing ALOT about it and it's <em>"Datawarehouse on your laptop"</em> mantra. However, the OTHER application that sometimes gets missed is <em>"SQLite for OLAP workloads"</em> and it was this concept that once I grasped it gave me a very interesting idea.... What if we could take the very pretty Aggregate Layer of our Data(warehouse/LakeHouse/Lake) and put that data right next to presentation layer of the lake, reducing network latency and... hopefully... have presentation reports running over very large workloads in the blink of an eye. It might even be fast enough that it could be deployed and embedded </p>
|
<li>Jaime Brian Head of Cloud Ninjas, TechConnect</li>
|
||||||
|
</ul>
|
||||||
|
<p><em>0422 012 17</em></p></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry><entry><title>Metabase and DuckDB</title><link href="http://localhost:8000/metabase-duckdb.html" rel="alternate"></link><published>2023-11-15T20:00:00+10:00</published><updated>2023-11-15T20:00:00+10:00</updated><author><name>Andrew Ridgway</name></author><id>tag:localhost,2023-11-15:/metabase-duckdb.html</id><summary type="html"><p>Using Metabase and DuckDB to create an embedded Reporting Container bringing the data as close to the report as possible</p></summary><content type="html"><p>Ahhhh <a href="https://duckdb.org/">DuckDB</a> if you're even partly floating around in the data space you've probably been hearing ALOT about it and it's <em>"Datawarehouse on your laptop"</em> mantra. However, the OTHER application that sometimes gets missed is <em>"SQLite for OLAP workloads"</em> and it was this concept that once I grasped it gave me a very interesting idea.... What if we could take the very pretty Aggregate Layer of our Data(warehouse/LakeHouse/Lake) and put that data right next to presentation layer of the lake, reducing network latency and... hopefully... have presentation reports running over very large workloads in the blink of an eye. It might even be fast enough that it could be deployed and embedded </p>
|
||||||
<p>However, for this to work we need some form of conatinerised reporting application.... lucky for us there is <a href="https://www.metabase.com/">Metabase</a> which is a fantastic little reporting application that has an open core. So this got me thinking... Can I put these two applications together and create a Reporting Layer with report embedding capabilities that is deployable in the cluster and has a admin UI accesible over a web page all whilst keeping the data locked to our network?</p>
|
<p>However, for this to work we need some form of conatinerised reporting application.... lucky for us there is <a href="https://www.metabase.com/">Metabase</a> which is a fantastic little reporting application that has an open core. So this got me thinking... Can I put these two applications together and create a Reporting Layer with report embedding capabilities that is deployable in the cluster and has a admin UI accesible over a web page all whilst keeping the data locked to our network?</p>
|
||||||
<h3>The Beginnings of an Idea</h3>
|
<h3>The Beginnings of an Idea</h3>
|
||||||
<p>Ok so... Big first question. Can Duckdb and Metabase talk? Well... not quite. But first lets take a quick look at the architecture we'll be employing here </p>
|
<p>Ok so... Big first question. Can Duckdb and Metabase talk? Well... not quite. But first lets take a quick look at the architecture we'll be employing here </p>
|
||||||
|
|||||||
@ -49,63 +49,84 @@ development and applying it in a strategic context.</p>
|
|||||||
<li>Containerised and Cloud Driven Data Architecture</li>
|
<li>Containerised and Cloud Driven Data Architecture</li>
|
||||||
</ul>
|
</ul>
|
||||||
<h1>EXPERIENCE</h1>
|
<h1>EXPERIENCE</h1>
|
||||||
<h2>Cloud Data Architect, Redeye Apps</h2>
|
<h2>Cloud Data Architect</h2>
|
||||||
<p><em>May 2022 - Present</em>
|
<h3><em>Redeye Apps</em></h3>
|
||||||
- Greenfields Research, Design and Deployment of S3 datalake (Parquet)
|
<h4><em>May 2022 - Present</em></h4>
|
||||||
- AWS DMS, S3, Athena, Glue
|
<ul>
|
||||||
- Research Design and Deployment of Catalog (Datahub)
|
<li>Greenfields Research, Design and Deployment of S3 datalake (Parquet)</li>
|
||||||
- Design of Data Governance Process (Datahub driven)
|
<li>AWS DMS, S3, Athena, Glue</li>
|
||||||
- Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)
|
<li>Research Design and Deployment of Catalog (Datahub)</li>
|
||||||
- CI/CD design and deployment of modelling and orchestration using Gitlab
|
<li>Design of Data Governance Process (Datahub driven)</li>
|
||||||
- Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy
|
<li>Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)</li>
|
||||||
- Design of ETL/Pipelines (DBT)
|
<li>CI/CD design and deployment of modelling and orchestration using Gitlab</li>
|
||||||
- Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</p>
|
<li>Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy</li>
|
||||||
<h2>Data Engineer, TechConnect IT Solutions</h2>
|
<li>Design of ETL/Pipelines (DBT)</li>
|
||||||
<p><em>August 2021 – May 2022</em>
|
<li>Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</li>
|
||||||
- Design of Cloud Data Batch ETL solutions using Python (Glue)
|
</ul>
|
||||||
- Design of Cloud Data Streaming ETL solution using Python (Kinesis)
|
<h2>Data Engineer,</h2>
|
||||||
- Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs
|
<h3><em>TechConnect IT Solutions</em></h3>
|
||||||
- Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)
|
<h4><em>August 2021 – May 2022</em></h4>
|
||||||
- Apply Prebuilt ML models to software solutions (Sagemaker)
|
<ul>
|
||||||
- Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)
|
<li>Design of Cloud Data Batch ETL solutions using Python (Glue)</li>
|
||||||
- API testing and development (gRPC, Rest)</p>
|
<li>Design of Cloud Data Streaming ETL solution using Python (Kinesis)</li>
|
||||||
<h2>Enterprise Data Warehouse Developer, Auto and General Insurance</h2>
|
<li>Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs</li>
|
||||||
<p><em>August 2019 - August 2021</em>
|
<li>Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)</li>
|
||||||
- ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL
|
<li>Apply Prebuilt ML models to software solutions (Sagemaker)</li>
|
||||||
- Bringing new data to the business to analyse for new insights
|
<li>Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)</li>
|
||||||
- Redeveloped Version Control and brought git to the data team
|
<li>API testing and development (gRPC, Rest)</li>
|
||||||
- Introduced python for API enablement in the Enterprise Data Warehouse
|
</ul>
|
||||||
- Partnering with the business to focus data project on actual need and translating into technical requirements</p>
|
<h2>Enterprise Data Warehouse Developer</h2>
|
||||||
<h2>Business Analyst, Auto and General Insurance</h2>
|
<h3><em>Auto and General Insurance</em></h3>
|
||||||
<p><em>January 2018 - August 2019</em>
|
<h4><em>August 2019 - August 2021</em></h4>
|
||||||
- Automate Service Performance Reporting using PowerShell/VBA/SAS
|
<ul>
|
||||||
- Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting
|
<li>ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL</li>
|
||||||
- Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends
|
<li>Bringing new data to the business to analyse for new insights</li>
|
||||||
- Technologies used include VBA, PowerShell, SQL, Web API’s, SAS
|
<li>Redeveloped Version Control and brought git to the data team</li>
|
||||||
- Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel
|
<li>Introduced python for API enablement in the Enterprise Data Warehouse</li>
|
||||||
- Gather Requirements to build meaningful reporting solutions
|
<li>Partnering with the business to focus data project on actual need and translating into technical requirements</li>
|
||||||
- Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</p>
|
</ul>
|
||||||
<h2>Forecasting and Capacity Analyst, Auto and General Insurance</h2>
|
<h2>Business Analyst</h2>
|
||||||
<p><em>January 2017 – January 2018</em>
|
<h3><em>Auto and General Insurance</em></h3>
|
||||||
- Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers
|
<h4><em>January 2018 - August 2019</em></h4>
|
||||||
- This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.
|
<ul>
|
||||||
- Forecast inbound workload requirements for the Auto and General sales call center using time series analysis
|
<li>Automate Service Performance Reporting using PowerShell/VBA/SAS</li>
|
||||||
- Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation
|
<li>Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting</li>
|
||||||
- Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy
|
<li>Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends</li>
|
||||||
- Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders
|
<li>Technologies used include VBA, PowerShell, SQL, Web API’s, SAS</li>
|
||||||
- Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</p>
|
<li>Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel</li>
|
||||||
<h2>Senior HR Performance Analyst, Queensland Department of Justice and Attorney General</h2>
|
<li>Gather Requirements to build meaningful reporting solutions</li>
|
||||||
<p><em>June 2016 - January 2017</em>
|
<li>Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</li>
|
||||||
- Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics
|
</ul>
|
||||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
<h2>Forecasting and Capacity Analyst</h2>
|
||||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
<h3><em>Auto and General Insurance</em></h3>
|
||||||
<h2>Workforce Business Analyst, Queensland Department of Justice and Attorney General</h2>
|
<h4><em>January 2017 – January 2018</em></h4>
|
||||||
<p><em>July 2015 – June 2016</em>
|
<ul>
|
||||||
- Develop and refine current workforce analysis techniques and databases
|
<li>Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers</li>
|
||||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
<li>This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.</li>
|
||||||
- Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit
|
<li>Forecast inbound workload requirements for the Auto and General sales call center using time series analysis</li>
|
||||||
- Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required
|
<li>Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation</li>
|
||||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
<li>Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy</li>
|
||||||
|
<li>Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders</li>
|
||||||
|
<li>Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</li>
|
||||||
|
</ul>
|
||||||
|
<h2>Senior HR Performance Analyst</h2>
|
||||||
|
<h3><em>Queensland Department of Justice and Attorney General</em></h3>
|
||||||
|
<h4><em>June 2016 - January 2017</em></h4>
|
||||||
|
<ul>
|
||||||
|
<li>Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics</li>
|
||||||
|
<li>Use VBA to automate regular reporting in Microsoft Access and Excel</li>
|
||||||
|
<li>Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</li>
|
||||||
|
</ul>
|
||||||
|
<h2>Workforce Business Analyst</h2>
|
||||||
|
<h3><em>Queensland Department of Justice and Attorney General</em></h3>
|
||||||
|
<h4><em>July 2015 – June 2016</em></h4>
|
||||||
|
<ul>
|
||||||
|
<li>Develop and refine current workforce analysis techniques and databases</li>
|
||||||
|
<li>Use VBA to automate regular reporting in Microsoft Access and Excel</li>
|
||||||
|
<li>Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit</li>
|
||||||
|
<li>Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required</li>
|
||||||
|
<li>Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</li>
|
||||||
|
</ul>
|
||||||
<h1>EDUCATION</h1>
|
<h1>EDUCATION</h1>
|
||||||
<ul>
|
<ul>
|
||||||
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
||||||
@ -113,8 +134,10 @@ development and applying it in a strategic context.</p>
|
|||||||
</ul>
|
</ul>
|
||||||
<h1>REFERENCES</h1>
|
<h1>REFERENCES</h1>
|
||||||
<ul>
|
<ul>
|
||||||
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health
|
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health</li>
|
||||||
<em>0428 038 031</em></li>
|
</ul>
|
||||||
<li>Jaime Brian Head of Cloud Ninjas, TechConnect
|
<p><em>0428 038 031</em></p>
|
||||||
<em>0422 012 17</em></li>
|
<ul>
|
||||||
</ul></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry></feed>
|
<li>Jaime Brian Head of Cloud Ninjas, TechConnect</li>
|
||||||
|
</ul>
|
||||||
|
<p><em>0422 012 17</em></p></content><category term="Resume"></category><category term="Cover Letter"></category><category term="Resume"></category></entry></feed>
|
||||||
@ -142,63 +142,84 @@ development and applying it in a strategic context.</p>
|
|||||||
<li>Containerised and Cloud Driven Data Architecture</li>
|
<li>Containerised and Cloud Driven Data Architecture</li>
|
||||||
</ul>
|
</ul>
|
||||||
<h1>EXPERIENCE</h1>
|
<h1>EXPERIENCE</h1>
|
||||||
<h2>Cloud Data Architect, Redeye Apps</h2>
|
<h2>Cloud Data Architect</h2>
|
||||||
<p><em>May 2022 - Present</em>
|
<h3><em>Redeye Apps</em></h3>
|
||||||
- Greenfields Research, Design and Deployment of S3 datalake (Parquet)
|
<h4><em>May 2022 - Present</em></h4>
|
||||||
- AWS DMS, S3, Athena, Glue
|
<ul>
|
||||||
- Research Design and Deployment of Catalog (Datahub)
|
<li>Greenfields Research, Design and Deployment of S3 datalake (Parquet)</li>
|
||||||
- Design of Data Governance Process (Datahub driven)
|
<li>AWS DMS, S3, Athena, Glue</li>
|
||||||
- Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)
|
<li>Research Design and Deployment of Catalog (Datahub)</li>
|
||||||
- CI/CD design and deployment of modelling and orchestration using Gitlab
|
<li>Design of Data Governance Process (Datahub driven)</li>
|
||||||
- Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy
|
<li>Research Design and Deployment of Orchestration and Modelling for Transforms (Dagster/DBT into Mesos)</li>
|
||||||
- Design of ETL/Pipelines (DBT)
|
<li>CI/CD design and deployment of modelling and orchestration using Gitlab</li>
|
||||||
- Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</p>
|
<li>Research, Design and Deployment of ML Ops Dev pipelines anddeployment strategy</li>
|
||||||
<h2>Data Engineer, TechConnect IT Solutions</h2>
|
<li>Design of ETL/Pipelines (DBT)</li>
|
||||||
<p><em>August 2021 – May 2022</em>
|
<li>Design of Customer Facing Data Products and deployment methodologies (Fully automated via Kakfa/Dagster/DBT)</li>
|
||||||
- Design of Cloud Data Batch ETL solutions using Python (Glue)
|
</ul>
|
||||||
- Design of Cloud Data Streaming ETL solution using Python (Kinesis)
|
<h2>Data Engineer,</h2>
|
||||||
- Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs
|
<h3><em>TechConnect IT Solutions</em></h3>
|
||||||
- Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)
|
<h4><em>August 2021 – May 2022</em></h4>
|
||||||
- Apply Prebuilt ML models to software solutions (Sagemaker)
|
<ul>
|
||||||
- Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)
|
<li>Design of Cloud Data Batch ETL solutions using Python (Glue)</li>
|
||||||
- API testing and development (gRPC, Rest)</p>
|
<li>Design of Cloud Data Streaming ETL solution using Python (Kinesis)</li>
|
||||||
<h2>Enterprise Data Warehouse Developer, Auto and General Insurance</h2>
|
<li>Solve complex client business problems using software to join and transform data from DB’s, Web API’s, Application API’s and System logs</li>
|
||||||
<p><em>August 2019 - August 2021</em>
|
<li>Build CI/CD pipelines to ensure smooth deployments (Bitbucket, gitlab)</li>
|
||||||
- ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL
|
<li>Apply Prebuilt ML models to software solutions (Sagemaker)</li>
|
||||||
- Bringing new data to the business to analyse for new insights
|
<li>Assist with the architecting of Containerisation solutions (Docker, ECS, ECR)</li>
|
||||||
- Redeveloped Version Control and brought git to the data team
|
<li>API testing and development (gRPC, Rest)</li>
|
||||||
- Introduced python for API enablement in the Enterprise Data Warehouse
|
</ul>
|
||||||
- Partnering with the business to focus data project on actual need and translating into technical requirements</p>
|
<h2>Enterprise Data Warehouse Developer</h2>
|
||||||
<h2>Business Analyst, Auto and General Insurance</h2>
|
<h3><em>Auto and General Insurance</em></h3>
|
||||||
<p><em>January 2018 - August 2019</em>
|
<h4><em>August 2019 - August 2021</em></h4>
|
||||||
- Automate Service Performance Reporting using PowerShell/VBA/SAS
|
<ul>
|
||||||
- Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting
|
<li>ETL development of CRM, WFP, Outbound Dialer, Inbound switch in Google Cloud, SAS, TSQL</li>
|
||||||
- Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends
|
<li>Bringing new data to the business to analyse for new insights</li>
|
||||||
- Technologies used include VBA, PowerShell, SQL, Web API’s, SAS
|
<li>Redeveloped Version Control and brought git to the data team</li>
|
||||||
- Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel
|
<li>Introduced python for API enablement in the Enterprise Data Warehouse</li>
|
||||||
- Gather Requirements to build meaningful reporting solutions
|
<li>Partnering with the business to focus data project on actual need and translating into technical requirements</li>
|
||||||
- Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</p>
|
</ul>
|
||||||
<h2>Forecasting and Capacity Analyst, Auto and General Insurance</h2>
|
<h2>Business Analyst</h2>
|
||||||
<p><em>January 2017 – January 2018</em>
|
<h3><em>Auto and General Insurance</em></h3>
|
||||||
- Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers
|
<h4><em>January 2018 - August 2019</em></h4>
|
||||||
- This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.
|
<ul>
|
||||||
- Forecast inbound workload requirements for the Auto and General sales call center using time series analysis
|
<li>Automate Service Performance Reporting using PowerShell/VBA/SAS</li>
|
||||||
- Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation
|
<li>Learn and leverage SAS EG and VA to streamline Microsoft Excel Reporting</li>
|
||||||
- Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy
|
<li>Identify and develop data pipelines to source data from multiple sources easily and collate into a single source to identify relationships and trends</li>
|
||||||
- Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders
|
<li>Technologies used include VBA, PowerShell, SQL, Web API’s, SAS</li>
|
||||||
- Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</p>
|
<li>Where SAS is inappropriate use VBA to automate processes in Microsoft Access and Excel</li>
|
||||||
<h2>Senior HR Performance Analyst, Queensland Department of Justice and Attorney General</h2>
|
<li>Gather Requirements to build meaningful reporting solutions</li>
|
||||||
<p><em>June 2016 - January 2017</em>
|
<li>Provide meaningful analysis on business performance and provide relevant presentations and reports to senior stakeholders.</li>
|
||||||
- Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics
|
</ul>
|
||||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
<h2>Forecasting and Capacity Analyst</h2>
|
||||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
<h3><em>Auto and General Insurance</em></h3>
|
||||||
<h2>Workforce Business Analyst, Queensland Department of Justice and Attorney General</h2>
|
<h4><em>January 2017 – January 2018</em></h4>
|
||||||
<p><em>July 2015 – June 2016</em>
|
<ul>
|
||||||
- Develop and refine current workforce analysis techniques and databases
|
<li>Develop the outbound forecasting model for the Auto and General sales call center by analysing the relationship between customer decisions and workload drivers</li>
|
||||||
- Use VBA to automate regular reporting in Microsoft Access and Excel
|
<li>This includes the complete data pipeline for the model from identifying and sourcing data, building the reporting and analysing the data and associated drivers.</li>
|
||||||
- Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit
|
<li>Forecast inbound workload requirements for the Auto and General sales call center using time series analysis</li>
|
||||||
- Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required
|
<li>Learn and leverage the Aspect Workforce Management System to ensure efficiency of forecast generation</li>
|
||||||
- Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</p>
|
<li>Learn and leverage the capabilities of SAS Enterprise Guide to improve accuracy</li>
|
||||||
|
<li>Liaise with people across the business to ensure meaningful, accurate analysis is provided to senior stakeholders</li>
|
||||||
|
<li>Analyse monthly, weekly and intraday requirements and ensure forecast is accurately predicting workload for breaks, meetings and Leave</li>
|
||||||
|
</ul>
|
||||||
|
<h2>Senior HR Performance Analyst</h2>
|
||||||
|
<h3><em>Queensland Department of Justice and Attorney General</em></h3>
|
||||||
|
<h4><em>June 2016 - January 2017</em></h4>
|
||||||
|
<ul>
|
||||||
|
<li>Harmonise various systems to develop a unified workforce reporting and analysis framework with appropriate metrics</li>
|
||||||
|
<li>Use VBA to automate regular reporting in Microsoft Access and Excel</li>
|
||||||
|
<li>Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</li>
|
||||||
|
</ul>
|
||||||
|
<h2>Workforce Business Analyst</h2>
|
||||||
|
<h3><em>Queensland Department of Justice and Attorney General</em></h3>
|
||||||
|
<h4><em>July 2015 – June 2016</em></h4>
|
||||||
|
<ul>
|
||||||
|
<li>Develop and refine current workforce analysis techniques and databases</li>
|
||||||
|
<li>Use VBA to automate regular reporting in Microsoft Access and Excel</li>
|
||||||
|
<li>Act as liaison between shared service providers and executives and facilitate communication during the implementation of a payroll leave audit</li>
|
||||||
|
<li>Gather reporting requirements from various business areas and produce ad-hoc and regular reports as required</li>
|
||||||
|
<li>Participate in government process through the production of briefs including Questions on Notice and Estimates Briefs for departmental executives</li>
|
||||||
|
</ul>
|
||||||
<h1>EDUCATION</h1>
|
<h1>EDUCATION</h1>
|
||||||
<ul>
|
<ul>
|
||||||
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
<li>2011 Bachelor of Business Management, University of Queensland</li>
|
||||||
@ -206,11 +227,13 @@ development and applying it in a strategic context.</p>
|
|||||||
</ul>
|
</ul>
|
||||||
<h1>REFERENCES</h1>
|
<h1>REFERENCES</h1>
|
||||||
<ul>
|
<ul>
|
||||||
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health
|
<li>Anthony Stiller Lead Developer, Data warehousing, Queensland Health</li>
|
||||||
<em>0428 038 031</em></li>
|
|
||||||
<li>Jaime Brian Head of Cloud Ninjas, TechConnect
|
|
||||||
<em>0422 012 17</em></li>
|
|
||||||
</ul>
|
</ul>
|
||||||
|
<p><em>0428 038 031</em></p>
|
||||||
|
<ul>
|
||||||
|
<li>Jaime Brian Head of Cloud Ninjas, TechConnect</li>
|
||||||
|
</ul>
|
||||||
|
<p><em>0422 012 17</em></p>
|
||||||
</article>
|
</article>
|
||||||
|
|
||||||
<hr>
|
<hr>
|
||||||
|
|||||||
Loading…
x
Reference in New Issue
Block a user