While you’re an agent in the act of killing lots of people by making a choice, your non-action does not mean that you escape moral responsibility. The moral responsibility lies in your ability to determine which is the rational act.
If the difference was between 1 person on one track and 2 on the other… the rational choice wouldn’t be to divert the train. The rational choice would be to seek ways to slow, derail or warn people to get out of the way. If you did all of these, would you then be exonerated from your moral responsibility?
You would know you tried everything to save everyone, that everyone mattered, and that the resulting deaths were not a consequence of your action or inaction, merely your inability to stop.
Of course, in the example provided, you would probably waste time trying to determine if an infinite number that was uncountable was bigger or smaller than an infinite countable number… so you’d end up checking out books, reading up on the morality of choice, and esoteric mathematics. In the final analysis, you’d probably miss the event horizon because you’d tried to solve riddle… and wound up checking references as the train started its murderous ride. You’d certainly be guilty of negligence on that score, but not culpability.
Of course, the evil genius that tied an infinite number of people to the tracks as well as the countable infinite number would be too busy to notice you pulling the lever… and anyway, wouldn’t they have rigged the lever so that it did the opposite of what you expect, or performed a random action, or was it out of order?