Yunus-bek Yevkurov was backed by 16 lawmakers, while one voted against and two ballots were declared invalid.
Russian President Dmitry Medvedev dismissed Zyazikov on Thursday, and appointed as acting leader Yevkurov, who had served as deputy head of the Volga-Urals Military District headquarters.
Zyazikov, 51, a former KGB officer and a close ally of Prime Minister Vladimir Putin, had served as president of Ingushetia, which neighbors with Chechnya, since 2002.
He has come under pressure in recent months over his failure to curb the surge in militant activity in the province. Human rights groups have also accused federal authorities of torture and abductions in the region.
In March this year, Zyazikov dissolved his government and called for major economic and social reforms.
In August, Ingushetia gained international media attention over the death of Magomed Yevloyev, a prominent journalist who had been involved in an argument on a plane with Zyazikov.
Yevloyev, who ran a banned website that had called for protests against the local government, was shot in the head while in a police vehicle, and died in hospital on August 31.
Yevkurov on Friday told Ingush lawmakers that he does not intend to make major personnel shifts in the local leadership.
