java - Spark JDBC SQLException -
i keep getting sqlexception suspect not problem. table :
create table person (first varchar(30) default null, last varchar(30) default null, gender char(1) default null, age tinyint(4) default null);
insert statements:
insert person values('barack','obama','m',54); insert person values('hillary','clinton','f',34);
spark code:
public static void main(string[] args) { sparkconf conf = new sparkconf().setappname("stackoverflow") .setmaster("local[4]"); javasparkcontext sc = new javasparkcontext(conf); sqlcontext sqlcontext = new sqlcontext(sc); map<string, string> options = new hashmap<>(); options.put("url", "jdbc:mariadb://localhost:3306/persondb"); options.put("user", "user"); options.put("password", "password333"); options.put("dbtable", "(select * person gender = 'm') someone"); dataframe jdbcdf = sqlcontext.read().format("jdbc"). options(options).load(); jdbcdf.show();
error:
exception in thread "main" org.apache.spark.sparkexception: job aborted due stage failure: task 0 in stage 0.0 failed 1 times, recent failure: lost task 0.0 in stage 0.0 (tid 0, localhost): java.sql.sqlexception: out of range value column 'age' : value age not in integer range
i tried changing table stmt(@jmj):
create table person (first varchar(30) default null, last varchar(30) default null, gender char(1) default null, age int default null);
then worked queries giving:
caused by: java.sql.sqlexception: out of range value column 'age' : value age not in integer range
the source of problem use of tinyint(4) storage age.
change type int insead tinyint(4).
to understand why check post.
hope helps.
Comments
Post a Comment