Let's start training the LDA model using the training corpus and keep track of the training time as follows:
val startTime = System.nanoTime()ldaModel = lda.run(corpus)val elapsed = (System.nanoTime() - startTime) / 1e9println("Finished training LDA model. Summary:")println("Training time: " + elapsed + " sec")
Now additionally, we can save the trained model for future reuse that can goes as follows:
//Saving the model for future useparams.ldaModel.save(spark.sparkContext, "model/LDATrainedModel")
Note that once you have finished the training and got the most optimal training, uncomment the preceding line before you deploy the model. Otherwise, it will get stopped by throwing an exception in the model reuse ...
