Fast Data Processing with Spark 2 - Third Edition

SparkContext - metadata

The SparkContext object has a set of metadata that I found useful. The version number, application name, and memory available are useful pieces of information. At the start of a Spark program, I usually display/log the version number.

Value	Use
appName	This value is the application name. If you have established a convention, this field can be useful at runtime.
getConf	This value returns configuration information.
getExecutorMemoryStatus	This value retrieves memory details. It could be useful if you want to check memory details. As Spark is distributed, the values do not mean that you are out of memory.
Master	This value is the name of the master.
Version	I found this value very useful, especially ...