We calculate the gravitational waves (GWs) induced by the density fluctuations due to inhomogeneous distribution of primordial black holes (PBHs) in the case where PBHs eventually dominate and reheat the universe by Hawking evaporation. The initial PBH density fluctuations are isocurvature in nature. We find that most of the induced GWs are generated right after evaporation, when the universe transits from the PBH dominated era to the radiation dominated era and the curvature perturbation starts to oscillate wildly. The strongest constraint on the amount of the produced GWs comes from the big bang nucleosynthesis (BBN). We improve previous constraints on the PBH fraction and find that it cannot exceed $10^{-4}$. Furthermore, this maximum fraction decreases as the mass increases and reaches $10^{-12}$ for $M{\rm PBH}\sim 5\times10^8 {\rm g}$, which is the largest mass allowed by the BBN constraint on the reheating temperature. Considering that PBH may cluster above a given clustering scale, we also derive a lower bound on the scale of clustering. Interestingly, the GW spectrum for $M{\rm PBH}\sim 10^4 -10^8 {\rm g}$ enters the observational window of LIGO and DECIGO and could be tested in the future. Although we focus on the PBH dominated early universe in this paper, our methodology is applicable to any model with early isocurvature perturbation.