What if I want a on demand or scheduled job to pull data from Kafka and insert it into StarRocks?
Of the 2 options:
Option 1: Routine Load
Option 2: Kafka Sink
I would use Routine Load. That allows you to submit a database job that will connect to Kafka (StarRocks is the client) and process the data in the queue. This means you will have to build a tool to have it pull data when you want it.