He was the son of Charles Neal and Olive Amanda (Lawhead) Barnes, born June 3, 1938, in Caroltown. Jim was raised by his aunt and uncle, Robert and Helen Huber.