In order to train the deep neural networks on Spark using DL4J, two primary wrapper classes need to be used:
- SparkDl4jMultiLayer: A wrapper around DL4J's MultiLayerNetwork
- SparkComputationGraph: A wrapper around DL4J's ComputationGraph
The network configuration process for the standard, as well as the distributed, mode remains same. That means, we configure the network properties by creating a MultiLayerConfiguration instance. The workflow for deep learning on Spark with DL4J can be depicted as follows:
Here are the sample code snippets for the workflow steps:
- Multilayer network configuration:
MultiLayerConfiguration conf ...
