Hi all
I’m trying to understand what is best practice when it comes to ingesting data into Starrocks from Kafka. I’d prefer to have the same strategy for all table types (if possible). Our data comes from Kafka and I’d like to ingest data into a few different table types - primary key tbl, duplicate key tbl and aggregate tbl. From the docs I can see that at least for the Aggregate tbl, the db is creating new versions for each batch and on read time all these versions are read (unless compaction has occurred).
Is that the same for the other table types?
Does that mean that my ingesting strategy should be in big batches, so that the number of versions is not that big?
What is the best practice to ingest from Kafka? Is there something I should pay attention to?