I tried convincing my manager but he argues it is not professional to leave the team early.
Since this is all within the same company this should not matter too much - everybody is supposed to cooperate for the good of the company. If the manager of team A wants to have you on the team, he could take it to whatever manager is in charge of teams A and B, and convince him to make it happen. Of course, there could be well-known internal rules that everybody is aware of about the circumstances under which teams can be changed - if those were against this, it might be a difficult case to make, and you may have to suck it up.