hadoop - How to store special characters in Hive? -

i've been playing around spark, hive , parquet, have data in hive table , here how looks ( warning french language ahead ) :

bloqu� � l'arriv�e      null probl�me de connexion   bloqu� en hub 

obviously there's wrong here.

what : read teradata table dataframe spark, store parquet file , use file store hive, here's create table script :

create table `table`(    `lib` varchar(255),    `libelle_sous_cause` varchar(255),    )  stored parquet  location    'hdfs://location'; 

i don't know cause this, might caused special encoding between teradata > parquet or parquet > hive, i'm not sure.

any appreciated, thanks.

i've run same problem when doing sqoop teradata hadoop. when extracting teradata, in select, please try wrapping varchar columns, may have issue, line:

select name, age, trim(cast(translate(columnofinterest using latin_to_unicode error) varchar(50))) tablename; 

columnofinterest column have special characters.

let me know if works.