Hi,
I have to create a query on several parquet file located on s3 storage ( minio ).
It works with : "SELECT * FROM FILE ( … ) " but I need to pass every time the s3 credential to FILE. With JOIN query, it is cumberstome, because you have to pass credential for each FILE clause.
CREATE TABLE bob AS
SELECT a, b FROM FROM FILES(
“path” = “s3://a.parquet”,
“format” = “parquet”,
“aws.s3.access_key” = “AAAAAAAAAAAAAAAAAAAA”,
“aws.s3.secret_key” = “BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB”
) LEFT JOIN FILES(
“path” = “s3://b.parquet”,
“format” = “parquet”,
“aws.s3.access_key” = “AAAAAAAAAAAAAAAAAAAA”,
“aws.s3.secret_key” = “BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB”
) `
It would be better to have something like duckdb, a secret engine to store credential or a s3 catalog
I would like to have something like :
SELECT * FROM FILE(“s3://bucket/test.parquet”)
Do you have other suggestion ?