Pupillary response has been widely accepted as a physiological index of cognitive workload. It can be reliably measured with remote eye trackers in a non-intrusive way. However, pupillometric measurement might fail to assess cognitive workload due to the variation of luminance conditions. To overcome this problem, we study the characteristics of pupillary responses at different stages of cognitive process when performing arithmetic tasks, and propose a fine-grained approach for cognitive workload measurement. Experimental results show that cognitive workload could be effectively measured even under luminance changes.