Ernest Henry Wemme (1861-1914) founded Willamette Tent & Awning Company in the 1880s in Portland, Oregon. In 1907, he sold the firm to brothers Max S. Hirsch (1871-1959) and Leopold Hirsch (1868-1961), who partnered with the firm's secretary, Henry Weis (1874-1940). By 1914, the company had renamed itself Hirsch-Weis Manufacturing Company. The company specialized in outdoor clothing and supplies.

In the 1930s, Max S. Hirsch's son, Harold S. Hirsch (1907-1990), began designing skiwear. He named this line of clothing White Stag, after the fact that the name "Weis" is similar to the German word for "white" (weiß), and the name "Hirsch" is the German word for "stag." The White Stag line became popular, and eventually expanded to other types of casual sportswear. In 1946, the company as a whole renamed itself White Stag Manufacturing Company.

The firm was purchased by Warner Brothers Company (later Warnaco) in 1966. Under Warnaco's management, White Stag was moved out of Portland. In 1986, financial difficulties led to a hostile takeover of Warnaco, and White Stag was spun off into part of a new company named Authentic Fitness Corporation. In 2003, Wal-Mart purchased the White Stag trademark.