java - Apache Spark - Unable to understand scala example -
i trying understand scala code on location. (i java background).
i feeling totally lost in below part
val pairs1 = sc.parallelize(0 until nummappers, nummappers).flatmap { p => val rangen = new random var arr1 = new array[(int, array[byte])](numkvpairs) (i <- 0 until numkvpairs) { val bytearr = new array[byte](valsize) rangen.nextbytes(bytearr) arr1(i) = (rangen.nextint(int.maxvalue), bytearr) } arr1 }.cache() i know parallelize , flat map . not getting how arr1 getting initialized . of type int or else- array of bytes ? also, doing inside loop logic.
var arr1 = new array[(int, array[byte])](numkvpairs) simply creates array of size numkvpairs , of type (int, array[byte]) (pair of int , array of bytes)
afterwards, arr1 filled random data.
Comments
Post a Comment