Have you unchecked “Out of stock visibility” in Woocommerce>Settings>Products>Inventory? If not, do it, and reindex all your products.