Welcome to Food/Drink Week at Milwaukee Record, brought to you by Third Space Brewing. From May 12 through May 19, belly up and enjoy adventurous, odd, and elaborate coverage of all things edible and drinkable in Milwaukee and beyond
Years ago, I was being interviewed by Milwaukee treasure Dori Zori. I don’t remember why Dori was interviewing me, but I’ll always remember her first question:
“What did your childhood smell like?”
It’s a great question. Smells, scents, and odors are powerful memory triggers, and can instantly take us back to specific times in our past. (For the record, my childhood smelled like brand new Nintendo games and the wood from a railroad bridge on the outskirts of town where my friends and I used to hang out.) But for the purposes of this article, let me pose a slightly different take on that question:
“What did your first few years in Milwaukee smell like?”
If, like me, you moved to Milwaukee in 1996 and lived in the UW-Milwaukee dorms for two years, the answer is simple: the trash chutes in the north tower of Sandburg Hall, and The Chocolate Factory. Ah yes, The Chocolate Factory.
Mike Toffler and Peter Blommer opened the first Chocolate Factory in Cedarburg in 1972. That location is still open today. According to its website, the subs-and-ice-cream shop currently has eight other locations in southeastern Wisconsin: West Bend, Menomonee Falls, Oconomowoc, Delafield, Pewaukee, Waukesha, Elm Grove, and Oak Creek.
You’ll notice that a prominent southeastern Wisconsin city is missing from that list: Milwaukee. But this wasn’t always the case. Back in 1996, The Chocolate Factory had a prime East Side location in the long-gone Prospect Mall. Remember the Prospect Mall? I do. It was on the corner of Prospect Avenue and Ivanhoe Place. It opened in 1976 and closed in 2006. The site is an apartment building now (natch) and a Potbelly location. Here’s what I wrote about the mall back in 2020:
When I arrived in Milwaukee in 1996, the Prospect Mall was a seedy den of sin and used books, a crumbling pleasure palace of Thai food and VHS porn. It was kind of creepy and gross. Maybe it had a heyday back in the ’80s, but I doubt it. It was a dump in 1996 and I suspect it was a dump in 1976. It smelled…weird. I loved it and I spent a ton of time there.
The two-screen movie theater was a beat-up but first-run kind of deal. The seedy video store (with porn) was Video Visions. The seedy used record/book store (with porn) was Recycled Books & Records. The Chocolate Factory was awesome. Thai Joe’s was Thai Joe’s. There was more, too: Bangkok Orchid (Thai food), Kosta’s White Manor (Greek food), Gothic Body (tattoos/piercings), and something called Drama Crow (no idea). Calling the whole thing a “mall” was a bit much—it was more like a long, dimly lit hallway with some shops on either side and some occasional half-dead plants in the middle. Maybe a bench or two.
God, I miss that mall. But let’s get back to The Chocolate Factory. Yes, there was a Chocolate Factory in the Prospect Mall, and for my first few years in Milwaukee, I ate there constantly. Here’s what I wrote about it, also in 2020:
Those subs. Those spices. That warm bread. I know The Chocolate Factory still exists in the Milwaukee area, but I haven’t been in one ever since the company pulled out of the Shops of Grand Avenue (R.I.P.) The Prospect Mall Chocolate Factory was a thing of beauty, all dark wooden booths and stained glass. I’m not sure if I’m misremembering, but, yeah, I’m pretty sure there was stained glass. It was like a church, if a church served turkey breast subs instead of the body of Christ and was located across the street from a shady print shop called Smithkin The Printer (later a Qdoba, now luxury apartments or whatever). Remember Smithkin The Printer? Anyone?
Like I said above, the Prospect Mall closed in 2006. There was a Chocolate Factory in the Shops of Grand Avenue after that, but that, too, closed a few years before the Grand Ave. was transformed into The Avenue in 2019. There hasn’t been a Milwaukee Chocolate Factory since.
And, more crucially, even though there are plenty of non-Milwaukee locations, I haven’t eaten at a Chocolate Factory in forever. At least 13 years. Maybe 15? I’m not exactly sure. But on a recent drive through Elm Grove, I changed that. I stopped in at The Chocolate Factory.
Would the smell of that warm bread instantly take me back to ’96-’98? Would the joys of a Chocolate Factory turkey breast sub make me forget that I was a complete and total doofus in ’96-’98? Would all my 2025 problems melt away like a pint of Chocolate Factory ice cream in the warm sun? Would I end up writing a heartfelt, heartening, and heartbreaking personal essay about wasted youth, the passage of time, and the gnawing feeling that it’s all moving so fast and that only decay and death await us at the finish line? Would this finally be the article I submitted to the Milwaukee Press Club, along with my $50 submission fee? Well…
No, not really. And that’s fine.
My turkey breast sub was good. The scent was good, too, but it didn’t really transport me. I had a nice meal, end of story. A lady glared at me while I took photos. I smiled back at her as I dug into a bag of sea salt potato chips. I hopped back in my car and drove away. The traffic was bad near State Fair Park. It’s always bad near State Fair Park. I wrote what you’re reading right now.
I moved to Milwaukee 29 years ago. I’ll be 48 soon. I still don’t know what I’m doing. All I know is that I/you/we can never go back. I suppose that’s enough. Thank you, Elm Grove Chocolate Factory. I mean it.
Want more Milwaukee Record? Subscribe to our free weekly newsletter and/or support us on Patreon.
RELATED ARTICLES