java - Apache Spark - Unable to understand scala example -


i trying understand scala code on location. (i java background).

https://github.com/apache/spark/blob/master/examples/src/main/scala/org/apache/spark/examples/groupbytest.scala

i feeling totally lost in below part

val pairs1 = sc.parallelize(0 until nummappers, nummappers).flatmap { p =>   val rangen = new random   var arr1 = new array[(int, array[byte])](numkvpairs)   (i <- 0 until numkvpairs) {     val bytearr = new array[byte](valsize)     rangen.nextbytes(bytearr)     arr1(i) = (rangen.nextint(int.maxvalue), bytearr)   }   arr1 }.cache() 

i know parallelize , flat map . not getting how arr1 getting initialized . of type int or else- array of bytes ? also, doing inside loop logic.

var arr1 = new array[(int, array[byte])](numkvpairs) 

simply creates array of size numkvpairs , of type (int, array[byte]) (pair of int , array of bytes)

afterwards, arr1 filled random data.


Comments