How do I use Dremio with Wasabi?

Dremio is validated for use with Wasabi. Dremio is a data lake engine that delivers analytics directly on your data lake storage.

Pre-Requisites

  • Active Wasabi cloud storage account
  • Operational Dremio server

Configuration

1. Login to the Dremio Server GUI

mceclip0.png

2. Click on Add Source to add Wasabi as a data source, as shown below.

mceclip1.png

3. Select Amazon S3 as shown below.

mceclip3.png

4. Before we enter the credentials (access and secret key) we have to make a couple of changes in the Advanced Options to enable S3 compatible support. Click on Advanced Options as shown below.

mceclip4.png

5. Check the 'Enable compatibility mode' box. Then, click on Add property.

mceclip5.png

6. Now, under Connection Properties, add fs.s3a.path.style.access and set the value to true

mceclip6.png

7. Now, add the fs.s3a.endpoint property and Wasabi service URL as value. Click here for more information on Wasabi service URLs. Click Save when done. 

mceclip7.png

8. Now, you can enter the name, and credentials for your Wasabi account in the General tab. Click Save when done.

mceclip8.png

9. At this point, the Dremio home page should display all your buckets, as shown below.

mceclip9.png

10. You can directly use the data within your buckets to run queries and analysis. For example, I have a .csv file in one of my buckets.

mceclip13.png

11. Clicking on the file displays the content for editing purposes. Hit Save when done.

mceclip11.png

12. The saved output can then be used to run analytics within the Dremio application interface.

mceclip12.png

 

Have more questions? Submit a request