Load csv with newlines in fields

Working with Data
1

Hi,
I have a csv file with newlines in fileds, for example:
“1”, “test text
here”, “2312”

During import, I get error Error: Value count does not match column count, I tried enclose ", still does’t work. I tried like this:

LOAD LABEL estates_test
(
    DATA INFILE("gs://test-bucket/all/snapshot/test.estates.*.csv")
    INTO TABLE estates
    COLUMNS TERMINATED BY ","
    FORMAT AS "CSV"
    (
       enclose = '"'
       escape = "\\"
    )
    (id, area, region, test_id)
)
WITH BROKER
(
	"gcp.gcs.service_account_email" = "***",
	"gcp.gcs.service_account_private_key_id" = "***",
	"gcp.gcs.service_account_private_key" = "-----BEGIN PRIVATE KEY-----\nM***\n-----END PRIVATE KEY-----\n"
)
PROPERTIES
(
    "timeout" = "3600"
);

In bigquery, for example it allows to import such csv files with option -allowQuotedNewlines, maybe you have some similar option ?

2

I haven’t seen one but doesn’t mean it might exist.

You can create a GitHub issue but this would very low in priority unless you are a commerical customer.

Your best alternative is to make a clean csv or use a 3rd party tool to make a clean csv.