To start the PySpark CLI shell, we just need the --master parameter to connect to the desired cluster. Based on the use case, the connection parameters would be changing along with the packages being used for connection. To connect to an Apache Cassandra cluster with authentication and SSL encryption enabled, the following list of parameters need to be passed to the PySpark shell. If any of them are not enabled on the Cassandra side, those parameters need to be removed accordingly. An example format for connecting to a Cassandra node with authentication and SSL enabled is as follows:
# To Start PYSpark shell without master or slave.# Remove authentication or ssl parameters based on cassandra side enabling$SPARK_HOME/bin/pyspark ...
