Spark mode - Other sources

SparkDataset constructor

StructType schema = DataTypes.createStructType(List.of(
                DataTypes.createStructField("string", DataTypes.StringType, false),
                DataTypes.createStructField("integer", DataTypes.LongType, false),
                DataTypes.createStructField("boolean", DataTypes.BooleanType, false),
                DataTypes.createStructField("float", DataTypes.DoubleType, false)
));

Dataset<Row> dataFrame = spark.createDataFrame(List.of(
        RowFactory.create("string", 1L, true, 1.5D)
), schema);


fr.insee.vtl.model.Dataset sparkDataset = new SparkDataset(dataFrame);

Other formats supported by Spark

See the official documentation