If your train is for example delayed, and you see that no one is getting on or off the train, there is no point in waiting any longer in my opinion. Also, it would be a bit more realistic if the train driver decides when to close the doors. I think that there should be a setting or a shortcut to skip the load passengers timer and just show the time your train has to leave the station.
In tsw the route with the shortest loading time is LIRR at 20 secconds, the bakerloo seems to be 10/15 most routes are 60s but some are only 30