armistace/blog
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

updates to content

Browse Source
This commit is contained in:
armistace
2023-10-28 10:34:43 +10:00
parent 236c1bec1e
commit d45891ffd9
6 changed files with 109 additions and 6 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/content/metabase_duckdb.md
+19 -1
View File
@@ -18,7 +18,25 @@ Ok so... Big first question. Can Duckdb and Metabase talk? Well... not quite. Bu
But you'll notice this pretty glossed over line, "Connector", that right there is the clincher. So what is this "Connector"?.
To Deep dive into this would take a whole blog so to give you something to quickly wrap your head around its the glue that will make metabase be able to query your data source.
To Deep dive into this would take a whole blog so to give you something to quickly wrap your head around its the glue that will make metabase be able to query your data source. The reality is its a jdbc driver compiled against metabase.
Thankfully Metabase point you to a [community driver](https://github.com/AlexR2D2/metabase_duckdb_driver) for linking to duckdb ( hopefully it will be brought into metabase proper sooner rather than later )
Now the release of this driver is still compiled against 0.8 of duckdb and 0.9 is the latest stable but hopefully the [PR](https://github.com/AlexR2D2/metabase_duckdb_driver/pull/19) for thi will land very soon giving a good quick way to link to the latest and greatest in duckdb from metabase
### But How do we get Data?
Brilliant, using the recomended DockerFile we can load up a metabase container with the duckdb driver pre built
```
FROM openjdk:19-buster
ENV MB_PLUGINS_DIR=/home/plugins/
ADD https://downloads.metabase.com/v0.46.2/metabase.jar /home
ADD https://github.com/AlexR2D2/metabase_duckdb_driver/releases/download/0.1.6/duckdb.metabase-driver.jar /home/plugins/
RUN chmod 744 /home/plugins/duckdb.metabase-driver.jar
CMD ["java", "-jar", "/home/metabase.jar"]
```
Great Now the big question. How do we get the data into the damn thing. Interestingly initially when I was designing this I had the thought of leveragin the in memory capabilities of duckdb and pulling in from the parquet on s3 directly as needed, after all the cluster is on AWS so the s3 API requests should be unbelievably fast anyway so why bother with a persistent database?