Can my mother be the karta of the HUF?

In your case, between you and your brother, if a new HUF is desired to be set up, the eldest son will be the karta of the HUF and not your mother