Japan’s Prime Minister Fumio Kishida reshuffled his cabinet on Wednesday, removing some ministers with links to the Unification Church in a bid to stem plunging support amid growing public outrage over the ruling party’s ties to the controversial group.

Kishida, in office since last October, announced his new government team in a shake-up that came earlier than analysts had expected.

While key personnel like foreign minister Yoshimasa Hayashi and finance chief Shunichi Suzuki held on to their posts, some high-profile ministers were removed, including Nobuo Kishi, the younger brother of slain former premier Shinzo Abe, replaced as defense minister by Yasukazu Hamada.