The Southern Charm matriarch opened up about her past loves like never before in her new book, The Art of Southern Charm, which is in stores now. Not only did she spill all the deets on her first husband and Whitney Sudler-Smith's father Lon Smith, but she also gave readers some insight into her second husband, Edward Fleming, and third husband, Arthur Altschul.
After Patricia and Lon divorced after 14 years of marriage, she "could have remarried several times, but being a good mother to Whitney always came first, my career came second, and having fun came third," she writes in The Art of Southern Charm. "I never dated when Whitney was in the house—it wasn't that important to me," Patricia says.
But then when Whitney went away to England and France, Patricia said she "started rethinking my priorities." A friend introduced her to Edward Fleming, a prominent psychiatrist who founded the Psychiatric Institutes of America in 1989. Patricia describes him in the book as "movie-star handsome, charming, an accomplished yachtsman," who was "a direct descendant of Robert E. Lee." "He quickly swept me off my feet and onto his magnificent new motor yacht, the Silver Cloud," Patricia writes.
Patricia and Edward were married, and they spent the next year-and-a-half sailing from place to place. But Patricia soon began to feel like she was cut off from the rest of the world while sailing around with Edward, and this pleasure cruise quickly hit some rough waters. "Ed tried to isolate me from everyone—and everything—I loved," Patricia writes in The Art of Southern Charm. "The coup de grâce? The moment that he proposed we renounce our American citizenship and move to Ireland for tax purposes... I decided it was time to get off the boat, even if it meant another divorce."
Patricia opens up about her life before Southern Charm in her new book, in stores now. Photo: Diversion Books
The Southern Charm grand dame then traveled back to Virginia to care for her ailing mother and to figure out what her next move would be. She ran into Arthur Altschul, one of her favorite former clients and a friend of hers for 20 years, at the reopening of James McNeill Whistler's Peacock Room at the Smithsonian in May 1993. "Ahhh...Arthur. He was the last of his breed: brilliant, charismatic, a great businessman, and a true gentleman with infallible taste and the best sense of humor," Patricia gushes in her book. "He was quite the luminary: a partner at Goldman Sachs, a philanthropist, a major art collector, and on top of everything else, a divine human being."
Patricia says that she felt like the two had always admired one another, but they were never single at the same time. But this time, Patricia was divorced, Arthur was divorced, and he invited her to come to New York for an event at the Metropolitan Museum of Art, where he served as a board member. Patricia accepted, and she "viewed our sudden romance as an adventure." The two eventually married and "settled into domestic bliss at a very high level," which included an apartment on Fifth Avenue in Manhattan, Overbrook Farm in Connecticut, and the waterfront estate Southerly in Oyster Bay, New York.
Unfortunately, after about five years of marriage, Arthur's health began to decline. "He was such a vital and vibrant man that he refused to allow illness to slow him down for a minute. He still enjoyed going out and having a very active social life," Patricia writes. "We never missed an opening at a museum, an auction house, or an art gallery, or a good party—at home or abroad."
Arthur passed away on St. Patrick's Day in 2002. "I was devastated to think that I would never see this wonderful man again," she writes. "We came to each other late in life and had so little time together. I was inconsolable."
Later in the book, Patricia writes that she has "four engagement rings from my past." She's ready to give those, as well as her mother's, to Whitney when he decides to settle down himself.
Check out more pics from Patricia's past, below.