Clay retorts and fire bricks were essential in allowing gas to burn as a light source, a new innovation in the mid-1800s. Prior to gas, candles and oil lamps were the only artificial light sources.
Fire bricks are used to line heat-intensive environments like kilns and furnaces. Clay retorts are airtight vessels used for burning coal to produce gas. The factory originally had three kilns, with chimneys rising from the south end of the building.
The factory produced fire clay products until the early 1930s. The next tenant was the American Molasses Company. After a vacancy in the mid-1980s, Red Hook developer Greg O'Connell bought and restored the building. It was home to the Carvart Glass Company, and is now vacant.
There's much more detail in the Landmark designation (see Related Resources for link).