Processing 1 TB with DuckDB in less than 30 seconds

And so can you

Get ready to toss out all the norms and conventional wisdom about distributed compute! Today, we are eradicating the belief that DuckDB can only be used for “small” data.

In this article, we will attack the following beliefs:

• Only Spark can be used for terabytes of data (or it is ALWAYS the best choice)

• You need a lot of time to process TBs of data

We want…