Shaped charge has been widely used for penetrating concrete. However, due to the obvious difference between the propagation of shock waves and explosion products in water and air, the theory governing the formation of shaped charge jets in water as well as the underwater penetration effect of concrete need to be studied. In this paper, we introduced a modified forming theory of an underwater hemispherical shaped charge, and investigated the behavior of jet formation and concrete penetration in both air and water experimentally and numerically. The results show that the modified jet forming theory predicts the jet velocity of the hemispherical liner with an error of less than 10%. The underwater jets exhibit at least 3% faster and 11% longer than those in air. Concrete shows different failure modes after penetration in air and water. The depth of penetration deepens at least 18.75% after underwater penetration, accompanied by deeper crater with 65% smaller radius. Moreover, cracks throughout the entire target are formed, whereas cracks exist only near the penetration hole in air. This comprehensive study provides guidance for optimizing the structure of shaped charge and improves the understanding of the permeability effect of concrete in water.