Vishnu is the second god in the Hindu triumvirate, which consists of three gods responsible for the creation, upkeep and destruction of the world.
The other two gods are Brahma and Shiva. Hinduism is a minority faith in Bangladesh, which is overwhelmingly Muslim.
The government’s cultural affairs adviser, Ayub Quadri, told the BBC he was thinking of resigning over the issue.
“Ultimately we at the ministry have to take responsibility,” he said.
Mr Quadri said the government had stepped up security measures on the border with India, after reports the stolen objects could be smuggled out of the country.
The French embassy in Dhaka condemned the theft.
“It is obviously an important loss and we have no doubt that fast, decisive and efficient measures will lead to finding the criminals,” charge d’affaires Jean Romnicianu told Reuters news agency.