Another important part of performing realistic testing is to use real data and real workloads. Synthetic data often won't exercise a system fully or find exotic edge cases. You can use fuzzing to generate random data, but if you have a production system, then you can simply use the data from this and parse your logs to generate a realistic workload to replay.
Obviously, don't replay actions onto a live system that could modify user data, and be wary of data privacy concerns or any action that could generate an external event, for example, sending a mass email or charging a credit card.
You can use a dedicated test system, but you still need to be careful to stub out any external APIs that don't have a test version.
Another ...
