Two refugee families, one island
The American L'HommeDieus begin with a marriage between two kinds of exile. Benjamin L'Hommedieu, a Huguenot born at La Rochelle about 1657, fled France around the time of the 1685 Revocation of the Edict of Nantes. Patience Sylvester, born on Shelter Island in 1664, was the daughter of Nathaniel Sylvester, an Amsterdam-raised English merchant, and Grizzell Brinley, daughter of a royal auditor who lost everything in the English Civil War. Their marriage tied the family to Sylvester Manor, and their son John L'Hommedieu (1707–1777) began the direct line that moved from Southold to Smithtown and on to the present day.
La Rochelle
Huguenot port. Home of the L'Hommedieu, Peron and Mouchard families.
Amsterdam
Giles Sylvester's merchant house. Nathaniel grows up among English Separatists.
Petworth & Datchet
Thomas Brinley marries Anne Wase; Grizzell's childhood amid civil war.
Virginia & Barbados
Nathaniel's tobacco voyage to Jamestown and the family sugar trade.
Shelter Island
Sylvester Manor; Benjamin and Patience marry and raise a family.
Southold to Smithtown
Five generations along the Nissequogue River.
How to use this site
Ancestor explorer lets you walk the tree one person at a time: click a parent to go back a generation, or search by name or place. The Bodiam Castle card under People and Places tests the family tree's claim of descent from the castle's builder. People and Places introduces the family's notable people, then a travel list of sites tied to the family, from Sussex to Minnesota.
A note on evidence
The tree data comes from the family's MyHeritage tree (exported September 2026). Links near the present and through the Sylvester and Brinley families are well documented. Some medieval links were copied between online trees and are marked here as unverified or contradicted where published sources disagree.
Stories
Arranged by era. Each story is marked Documented when it rests on records or published history, or Partly traditional when part of it depends on tradition or a historian's reconstruction. Needs checking means the family tree itself has a link that has to be confirmed.
Sylvester Manor: the house that kept the story
People come and go in a family history. Sylvester Manor stays. For more than three and a half centuries it has been the fixed point the Sylvesters, L'Hommedieus, Derings, Gardiners, Horsfords and Fiskes kept returning to: the place where the family was founded, where its two main branches met, where its fortunes were lost and bought back, and where its hardest history is now told in the open. It is less a backdrop than a supporting character, and it has a scene in almost every era of this site.
Visit sylvestermanor.org →The manor on the Sylvester family →
A plantation on Manhansett land
Nathaniel Sylvester and his partners take up Shelter Island as a provisioning plantation for the family's Barbados sugar trade. Nathaniel settles here with Grizzell Brinley after their 1653 marriage. The wealth that built the manor came from the labor of enslaved Africans and Native people, and the manor today tells that part of its history first.
A refuge for Quakers
While Massachusetts was banishing, whipping and hanging Friends, the Sylvesters took in the banished Southwicks and in 1672 hosted George Fox. The manor helped the young Quaker movement survive, even as the household held people in bondage. See the Stories tab for this paradox in full.
Where the two branches meet
Patience Sylvester, raised on the manor, marries the Huguenot refugee Benjamin L'Hommedieu. Every American L'HommeDieu in Russ's line, and Ezra L'Hommedieu's line of cousins, starts with that marriage.
Brinley's Georgian house
Brinley Sylvester builds the Manor House that still stands. His daughter Mary sits for Joseph Blackburn in 1754, the portrait now at the Met. Two years later she marries Thomas Dering, and the manor passes into the Dering line. Follow Mary's lineage →
War, debt and an auction
The Derings flee to Middletown, Connecticut, during the Revolution. Their heirs later fall into debt, and in 1827 the manor is auctioned at Sag Harbor. The buyers are Ezra L'Hommedieu's daughter Mary Catherine and her husband Samuel Smith Gardiner, a Sylvester descendant through Patience. The L'Hommedieu branch buys the family seat back.
A summer house for poets
Mary Catherine's daughters marry the Harvard chemist Eben Norton Horsford, whose baking powder paid for Viking monuments in Boston. He makes the manor a summer retreat for Longfellow, Whittier, Lowell and other Cambridge writers.
Miss Alice's garden
Andrew and Alice Fiske, tenth-generation descendants of Nathaniel and Grizzell, keep the house, windmill and grounds standing. Alice's garden becomes famous on its own. Ezra's gold watch, his copy of Common Sense, and Ralph Earl's proof of Ezra's portrait stay at the manor.
Given to the public
Eben Fiske Ostby inherits the manor, gives the family archive to New York University, and with his nephew Bennett Konesni helps found the Sylvester Manor Educational Farm. In 2014, after 363 years in one family, the property passes to the nonprofit. It becomes a National Register Historic District in 2015.
A working farm that tells the whole story
236 acres with the 1737 Manor House, the restored windmill and the Afro-Indigenous Burial Ground. A working farm that sells produce and gives food to local families, public trails, Manor House tours, free field trips for Shelter Island schoolchildren, a Juneteenth commemoration, and Sculpture @ Sylvester Manor. Grounds are open March to October.
Sources: Mac Griswold, The Manor (2013); Sylvester Manor, sylvestermanor.org; NYU Fales Library, Sylvester Manor Archive. Details for each era are cited on the Stories and Sources tabs.
Mary Sylvester's lineage
Mary Sylvester (1725–1794) was the great-granddaughter of Nathaniel Sylvester and Grizzell Brinley, and Ezra L'Hommedieu's second cousin. Their fathers, Brinley Sylvester and Benjamin L'Hommedieu II, were first cousins, both born in 1694. By our count from the family tree, Mary is Russ's second cousin, seven times removed.
Her parents: Brinley Sylvester (1694–1752), born in East Hampton and raised in Rhode Island as a merchant. He inherited the manor from his uncle Giles after a legal dispute and built the Georgian house that still stands. He married Mary Burrough; they had two daughters.
Her sister: Margaret (1719–1782), who married David Chesebrough in 1749 and died at Stonington, Connecticut. Blackburn painted her the same year as Mary, and her portrait is also at the Met.
Her marriage: In 1756, at Newport, she married Thomas Dering, a Boston merchant and son of Henry Dering and Elizabeth Packer. When his business failed in the 1760s the family moved to the manor, and during the Revolution they fled to Middletown, Connecticut.
Her children: Sylvester Dering, who managed the manor and later appears as a brigadier general; Henry Packer Dering (1763–1822), Sag Harbor's first customs master, who married Anna Fosdick in 1793; and Elizabeth (Betsey), who married Nathaniel Gardiner. An infant, Henry Sylvester Dering, lived from April 1757 to January 1758. A later Sylvester Dering gave the portrait to the Met in 1916, so it stayed in her direct line for more than 160 years.
Two branches from one couple
Both lines start with Nathaniel Sylvester and Grizzell Brinley, whose own parents were Thomas Brinley, an auditor to Charles I, and Anne Wase.
- Sylvester branch
- Nathaniel Sylvester (1610–1680) + Grizzell Brinley
- Nathaniel Sylvester II + Margaret Hobart (wife needs checking)
- Brinley Sylvester (1694–1752) + Mary Burrough
- Mary Sylvester (1725–1794) + Thomas Dering
- Sylvester, Henry Packer and Elizabeth Dering
- L'Hommedieu branch
- Nathaniel Sylvester (1610–1680) + Grizzell Brinley
- Patience Sylvester (1664–1719) + Benjamin L'Hommedieu
- Benjamin L'Hommedieu II (1694–1755)
- Ezra L'Hommedieu (1734–1811)
- Mary Catherine L'Hommedieu + Samuel Smith Gardiner
The branches came together again over the manor. The Derings held it until their heirs sold it at auction in 1827, and the buyers were Ezra's daughter Mary Catherine and Samuel Smith Gardiner. Russ's own line runs through Patience's son John L'Hommedieu (1707–1777), Ezra's uncle. Benjamin also had a brother, John, naturalized in New York in 1687, so not every American L'Hommedieu descends from Patience.
Where the sources disagree
- Birth year: the Met gives 1725; the East Hampton Star gives 1724.
- Marriage date: a University of Michigan finding aid says 1738, which is almost certainly wrong, since she would have been 13. The Met gives 1756.
- Generation count: the NYU archive calls her Nathaniel's great-great-granddaughter. Nathaniel, then Nathaniel II, then Brinley, then Mary makes her a great-granddaughter.
- Grizzell's father: some sources name Francis Brinley. Francis was her brother; Thomas was her father.
Only the Met page was opened directly for this section. The other details come from the NYU Sylvester Manor Archive finding aid, the East Hampton Star, WikiTree, Find a Grave, Wikipedia and Nutfield Genealogy as summarized in research notes, and have not yet been checked page by page. The Research notes tab tracks them.
Ancestor explorer
Click any parent card to move back a generation. The line tags show which family branch each person belongs to. The L'HommeDieu line runs thirteen generations, from Dr. Russ back to Pierre L'Hommedieu (b. about 1575); the three French generations before Benjamin are marked unverified.
Is Bodiam Castle in the family?
Bodiam Castle was built from 1385 by Sir Edward Dalyngrigge (c.1346–1393), a veteran of the Hundred Years' War, after Richard II licensed him to fortify his manor. He held Bodiam through his wife Elizabeth Wardedieu, heiress of the family that had owned the manor since before 1330. The moated castle, with its round corner towers, twin-towered gatehouse and surviving portcullis, has belonged to the National Trust since Lord Curzon left it to the Trust in 1925.
The family's MyHeritage tree places Sir Edward and Elizabeth twenty generations back, as 18×-great-grandparents of Russ, through a Sussex chain that ends with Anne Wase, Grizzell Brinley's mother. Each link below is rated against published sources.
Verdict: a Sussex connection, but not yet a castle
The lower half of the chain is solid: Anne Wase of Petworth really is Grizzell Brinley's mother, and so a direct ancestor. The Dawtrey generations also appear in published pedigrees. The weak point is the very first step. Standard histories give Sir Edward one heir, his son Sir John, who died childless in 1408, after which Bodiam passed through a cousin to the Lewknor family. The published Dawtrey pedigree names John Dawtrey's wife as Joan Scardeville, not Joan Dalyngrigge. Until a record shows otherwise, treat Bodiam as a place from your ancestors' Sussex world rather than proven ancestry.
Where to look next: the Heralds' Visitations of Sussex (published by the Harleian Society), the History of Parliament biographies of the Dawtrey and Ernley men, and Dawtrey and Barttelot family papers at the West Sussex Record Office.
People and Places
People and sites connected to the family, places in rough order of the family's journey. Links go to pages checked while building this site.
Timeline
Research notes
Places where the sources disagree, and what would settle each one. Fix these in the MyHeritage tree before publishing the site more widely.
| Question | What the sources say | Next step |
|---|
Relatives worth knowing
These people are not in the direct line, but they are close kin and central to the family story.
Sources
Every web link below was opened and checked in September 2026.
Books and archives
- Mac Griswold, The Manor: Three Centuries at a Slave Plantation on Long Island (Farrar, Straus and Giroux, 2013).
- Henry B. Hoff, "The Sylvester Family of Shelter Island," New York Genealogical and Biographical Record 125 (1994).
- Sylvester Manor Archive, Fales Library & Special Collections, New York University (Record Group II: L'Hommedieu Family).
- Family MyHeritage tree, "L'HommeDieu Family Tree," GEDCOM export of 23 September 2026 (983 people). View the tree online.
- Russ L'HommeDieu, "History of the L'HommeDieu / L'Hommedieu Family and Their Relationship to Shelter Island" (family research document, 2025).