Determining the most popular non-stop flights

Expanding upon our tripGraph GraphFrame, the following query will allow us to find the most popular non-stop flights in the US (for this dataset):

# Determine the most popular non-stop flights
import pyspark.sql.functions as func
topTrips = tripGraph \
  .edges \
  .groupBy("src", "dst") \
  .agg(func.count("delay").alias("trips"))

# Show the top 20 most popular flights (single city hops)
display(topTrips.orderBy(topTrips.trips.desc()).limit(20))

Note, while we are using the delay column, we're just actually doing a count of the number of trips. Here's the output:

Determining the most popular non-stop flights

As can be observed from this query, the two most ...

Get Learning PySpark now with the O’Reilly learning platform.

O’Reilly members experience books, live events, courses curated by job role, and more from O’Reilly and nearly 200 top publishers.