The cost function is the key to solving any problem using optimization, and it's usually the most difficult thing to determine. The goal of any optimization algorithm is to find a set of inputs—flights, in this case—that minimizes the cost function, so the cost function has to return a value that represents how bad a solution is. There is no particular scale for badness; the only requirement is that the function returns larger values for worse solutions.
Often it is difficult to determine what makes a solution good or bad across many variables. Consider a few of the things that can be measured in the group travel example:
The total price of all the plane tickets, or possibly a weighted average that takes financial situations into account.
The total time that everyone has to spend on a plane.
Time spent at the airport waiting for the other members of the party to arrive.
Flights that leave too early in the morning may impose an additional cost by requiring travelers to miss out on sleep.
If the party rents a car, they must return it earlier in the day than when they rented it, or be forced to pay for a whole extra day.
It's not too hard to think of even more aspects of a particular schedule that could make the experience more or less pleasant. Any time you're faced with finding the best solution to a complicated problem, you'll need to decide what the important factors are. Although this can be difficult, the big ...