My gut says the opposite - I would think the second is faster if you have an index on primaryColumn. The execution plans would be different for both scenarios.
First: 1) Retrieve all rows from disk (sorting by primaryColumn if no index is returned) 2) Return the first one across the network 3) Hope the db is smart enough to retrieve all the rows before discarding them (many are not)
Second: 1) Go directly to the max row via a sorted index 2) Retrieve the one row from disk 3) Return the one row across the network
As you can see, it is possible to speculate that either is faster. Paul and I bother have different gut instincts. All that matters is what is faster for you of course. Which means you'll have to try it.
I'm going to go back and sec and say you really need to run tests on real data (not test data). Query optimizes are magically creatures that may or may not use indexes properly. In other words, while the proper indexes should guarantee a near-optimal query path, there's no guarantee a particular DBMS system will find the path; since most query optimizes do not have the time to search all paths.
The only piece of advice I can give you is to avoid nested queries. In general, query optimizes make better use of search trees with joins rather than nested queries, although in the case of max/min, I'm not sure you can avoid the subquery other than sorting your results.
Also, some systems support a third query "SELECT TOP 1 * FROM table order by primaryColumn desc".