Volumes serving Cassandra data should be formatted with the XFS filesystem. If you are not able to use XFS, then the EXT4 filesystem is a good alternative. ZFS also has a good following in the Cassandra community, but it is currently not supported by most enterprise Linux vendors.
BTRFS should be used with caution (Apache, 2016), and EXT3 is too slow to warrant consideration.