Comparing Sephardim to African Americans, former minister says the time has come to stop voting for “white people”