The point ambiguity owing to the ambiguous local appearances of image points is the one of the main causes making the stereo problem difficult. Under the point ambiguity, local similarity measures are easy to be ambiguous and this results in false matches in ambiguous regions. In this paper, we present the new similarity measure to resolve the point ambiguity problem based on the idea that the distinctiveness, not the interest, is the appropriate criterion for the feature selection under the point ambiguity. The proposed similarity measure named the Distinctive Similarity Measure (DSM) is essentially based on the distinctiveness of image points and the dissimilarity between them, which are both closely related to the local appearances of image points; the distinctiveness of an image point is related to the probability of a mismatch while the dissimilarity is related to the probability of a good match. We verify the efficiency of the proposed DSM by using testbed image sets. Experimental results show that the proposed DSM is very effective and can be easily used for improving the performance of existing stereo methods under the point ambiguity.