In general, you're right about making comparisons across teams -- one team's point may not be the same as another team's point. If the point sizes are consistent across teams, which it sounds like it may be in your case, then maybe it's useful to compare across teams. What does management do with those metrics though? What they do with them also plays into whether or not the metrics tracking makes sense. If it's just to see if one team is more productive than another, then I would say that's not the way it should be. Teams are made up of people, not robots. Different people will have different rates of solving problems. Heck, depending on the prevailing situation, the same person may not even solve solve the same problem the same way every time. There are many variables, both internal and external, to the person/people working on the problem so using story points to measure productivity is not encouraged.
The advice that I pass on to teams is to care more about your point
trends rather than the point totals. If you completed an average of 10 story points in the last three iterations and then only do 5 points in this iteration, then something might be going on that the team or the scrum master or product owner needs to address. Point estimate metrics should be used to help spot problems and make improvements.
Remember too that points are used for estimation and estimates are imprecise by nature. That's another red flag, when points start getting used as precise measures. Some teams even abandon story point estimation after a while because of the problems and chaos involved when people can't agree on how to use them properly. In fact, there's actually a growing push for
#NoEstimates -- you just break stories up into manageable chunks and iterate.