Pandas, Numpy and StarRocks

Working with Data Data Loading Tools & Integrations
1

Using the allin1 container.

Here’s an example with only Pandas

atwong@Alberts-MBP-3 sandbox % cat script.py
# import necessary packages
import pandas as pd
from sqlalchemy import create_engine

# establish connection with the database
engine = create_engine(
    "mysql+mysqlconnector://root:@localhost:9030/demo")

# read table data using sql query
sql_df = pd.read_sql(
    "SELECT * FROM sr_member",
    con=engine
)

print(sql_df.head())

atwong@Alberts-MBP-3 sandbox % python3 script.py
   sr_id     name  city_code    reg_date  verified
0      1      tom     100000  2022-03-13         1
1      2  johndoe     210000  2022-03-14         0
2      3   maruko     200000  2022-03-14         1
3      5   pavlov     210000  2022-03-16         0
4      4  ronaldo     100000  2022-03-15         0

Here’s another example with Numpy.

import mysql.connector
import numpy as np
import pandas as pd

mydb = mysql.connector.connect(
  host="localhost",
  port=9030,
  user="root",
  password="",
  database="demo"
)

mycursor = mydb.cursor()
mycursor.execute("SELECT * FROM sr_member")
myresult = mycursor.fetchall()

data = np.array(myresult)
sql_df = pd.DataFrame(data)

print(sql_df.head())
print(sql_df)

Results

atwong@Alberts-MBP-3 sandbox % python script.py
   0        1       2           3  4
0  5   pavlov  210000  2022-03-16  0
1  2  johndoe  210000  2022-03-14  0
2  3   maruko  200000  2022-03-14  1
3  1      tom  100000  2022-03-13  1
4  4  ronaldo  100000  2022-03-15  0
   0         1       2           3  4
0  5    pavlov  210000  2022-03-16  0
1  2   johndoe  210000  2022-03-14  0
2  3    maruko  200000  2022-03-14  1
3  1       tom  100000  2022-03-13  1
4  4   ronaldo  100000  2022-03-15  0
5  6  mohammed  300000  2022-03-17  1