Working on Polybase implementation where data stored as parquet file has to be imported in Synapse (SQL DWH).
So based on number of objects, created respective polybase(s). After executing few of them successfully, I got below error for one of them
To troubleshoot checked the error statement ---
HdfsBridge::recordReaderFillBuffer - Unexpected error encountered filling record reader buffer: MalformedInputException: Input length = 2
Why it happened
I have created the external table with varchar as the datatype for all the columns so that all datatypes can be handled.
CREATE EXTERNAL TABLE SampleTbl(
ID VARCHAR(200),
NOTE VARCHAR(1000),
ROWGUID VARCHAR(200),
CreateDate VARCHAR(200)
)
But there's an exception, columns having datetime values and that's the reason for the error.
Although external table will be created, the error will come while inserting in destination table, where expected is datetime value but trying to be inserted is varchar.
The error message doesn't say so 😉
What to do
So while creating external table, check for columns having datetime datatype. If there is then specify it .
CREATE EXTERNAL TABLE SampleTbl(
ID VARCHAR(200),
NOTE VARCHAR(1000),
ROWGUID VARCHAR(200),
CreateDate DATETIME
)