Model personalization is a key aspect for biophysical models to impact clinical practice, and cardiac contractility personalization from medical images is a major step in this direction. Existing gradient-based optimization approaches show promising results of identifying the maximum contractility from images, but the contraction and relaxation rates are not accounted for. A main reason is the limited choices of objective functions when their gradients are required. For complicated cardiac models, analytical evalua-tions of gradients are very difficult if not impossible, and finite difference approximations are computationally expensive and may introduce numerical difficulties. By removing such limitations with derivative-free optimization, we found that a velocity-based ob-jective function can properly identify regional maximum contraction stresses, contraction rates, and relaxation rates simultaneously with intact model complexity. Experiments on synthetic data show that the parameters are better identified using the velocity-based objective function than its position-based counterpart, and the proposed framework is insensitive to initial parameters with the adopted derivative-free optimization algorithm. Experiments on clinical data show that the framework can provide personalized contractility parameters which are consistent with the underlying physiologies of the patients and healthy volunteers.