The history of the garden begins in the early nineteenth century when the riverfront property was developed as a garden for the mansion of the wealthy Katz family, designed by the architect Friedrich Weinbrenner and completed in 1813. The Katz family that gave the garden its name has been one of the most influential families in the history of Gernsbach for centuries, having acquired great wealth in the business of shipping wood from the Black Forest into the Rhine valley and then up the Rhine as far as the Netherlands. Somewhat confusingly, the name of the other family that traditionally made up the town's aristocracy together with the Katz was Kast, and at times the two families intermarried. To this day, both names can be found all over the older parts of town. Throughout much of the 19th century as well as the first half of the 20th, the Katz family decorated the garden with a growing collection of statues and other stone and and metal decorative elements, some as old as the 15th century. They also planted trees rare at the times of which some survive. A large bald cypress (Taxodium distichum) dominates one part of the garden and two large magnolias (Magnolia x soulangeana) were already mentioned in a travel guide in the 19th century and were then considered to be among the largest magnolias in Germany. In the 1960s the Katz'scher Garten was opened to the public, but in the following decades it deteriorated until it was all but ruined and overgrown by the early 1990s.
In 1995 a initiative to restore the garden was founded by local residents and from 1996 until 2001 the garden was thoroughly renovated, restoring many of the playful intricacies of the 19th century designs and many of the antique ornaments that once graced the garden. The replanting of the garden was undertaken with many of the rare or exotic species now seen and since then many more have been added. Interestingly, the restored garden might even have had an effect on the gardening tastes of others in town for plants like Japanese fiber bananas (Musa basjoo) and cypresses are becoming ever more common throughout the area.