He held numerous NWA World Tag Team Championships with Gene Anderson who was portrayed as his brother.

Anderson started wrestling in 1967 in the American Wrestling Association (AWA) as Rock Rogowski, where he held the AWA Midwest Heavyweight and the AWA Midwest Tag Team Titles.

He went on to the National Wrestling Alliance where he adopted the ring name Ole Anderson, and formed what became a legendary tag team called the Minnesota Wrecking Crew with his "brother" Gene Anderson. They feuded with Mr. Wrestling & Mr. Wrestling II, Wahoo McDaniel, Jack Brisco, Jerry Brisco, Tommy Rich and Thunderbolt Patterson throughout the 1970s and early 1980s.

Behind the scenes, Anderson was also the primary booker for Georgia Championship Wrestling. When owners Jack and Jerry Brisco sold the promotion to Vince McMahon, Anderson resisted the change, later starting his own promotion.

In April 1985, Anderson was teaming with Thunderbolt Patterson when Arn Anderson debuted in the NWA. Anderson soon turned on Thunderbolt to team with Arn as the Minnesota Wrecking Crew.

In 1986, Anderson became part of the original Four Horsemen, a heel stable, with Ric Flair, Arn Anderson, Tully Blanchard, and manager James J. Dillon. During his time in the Horsemen, Anderson feuded with Magnum T.A., Dusty Rhodes, The Rock 'N Roll Express and the Road Warriors. Anderson was later kicked out of the group in favor of Lex Luger in early 1987.

Anderson retired in 1988, when his son, Brian, was starting his own amateur wrestling career. Brian would wrestle later in WCW as Bryant Anderson.

Anderson returned to wrestling with WCW in 1989 to reform the Four Horsemen with Flair, Arn and Sting. They quickly kicked Sting out of the group, and Anderson retired again to manage the Horsemen, who by then also included Barry Windham and Sid Vicious.

By 1990, Anderson had been chosen to head the booking committee for WCW, which was at that time beginning to phase out the use of the NWA name on its television programming. Appearing in the credits for WCW Pay-per-views (PPVs) under his real name, Anderson was responsible for some of the more infamous creative ideas tried by WCW. Among his creations were The Black Scorpion, which was intended to be a nemesis from Sting's past. The poorly-conceived gimmick (originally voiced by Anderson) was intended to be a vehicle for bringing in The Ultimate Warrior, but Warrior had no interest in leaving the World Wrestling Federation (WWF). After several miscues, the Scorpion's identity was eventually revealed as Ric Flair, in a ploy to confuse Sting and force him to lose the WCW World Heavyweight Championship.

Anderson briefly appeared on WCW TV in 1993 during a short-lived Four Horsemen reunion, but quickly disappeared from television soon after. He was fired from the company by new WCW head Eric Bischoff when he tried to get his son a job in Smoky Mountain Wrestling after his son got fired by Bischoff while training at the WCW Power Plant and letting Smoky Mountain head Jim Cornette on WCW property to consult with him. He is now mostly retired from wrestling, although he does make frequent appearances as wrestling conventions and other gatherings.