We present the results from two experiments, one from children and one from adults, that address the mechanisms by which language might help to shape color categorization. In the first experiment, the categorical advantage in discrimination and memory for colors that cross linguistic boundaries was attenuated in abnormal language development. Deaf and autistic individuals showed less reliance on categorical information than their typically developing controls. In the second experiment, we demonstrate that Korean (but not English) speakers show Categorical Perception (CP) on a visual search task for a boundary between two Korean color categories that is not marked in English. These effects were observed regardless of whether target items were presented to the left or right visual field. Dividing Korean participants into fast and slow responders demonstrated that fast responders show CP only in the right visual field while slow responders show CP in both visual fields. We argue that this finding is consistent with the view that CP in both hemispheres is verbally mediated by the left hemisphere language system.