The valence band structure and optical transitions of holes are investigated for elongated InGaAs/GaAs quantum dot (QD). We use the 4×4 Luttinger Hamiltonian under effective mass approximation for the estimation of the valence band structure. The energy eigenvalues and corresponding eigenvectors have been calculated by numerical diagonalization of Hamiltonian for lens-shaped QD using the harmonic oscillator basis function without considering the strain effect. We analyzed the impact of the size and composition of the QD on valence band structure and the transition probability of holes from ground state to excited states. The analysis reveals that the hole energy states form the energy band of heavy-hole (hh) and light-hole (lh) states. The band mixing of hh and lh energy states is decreasing with an increase in the lateral dimension (LD) of the QD which is indicated by the increase in the hh-lh band offset (Boff) parameter. The hh and lh states show strong intermixing for the small values of indium (In) concentration less than 0.23 and for the LD less than 10 nm for which the Boff parameter is found to be less than 25.22 meV. The widths of hh and lh bands are found to decrease with an increase in LD, and it is increasing with In concentration of the QD. The transitions from hh to hh states are polarization-sensitive, but transitions from hh to lh states are insensitive to the polarization state of the incident photon. The square optical matrix elements are decreasing with LD and increasing with In concentration.