This castle which stood on the site of warehouses on the side of the river Thames, and opposite Benet's-hill, was built by Baynard, a Norman adventurer who came over with William the Conqueror. He received many marks of that king's favour, and obtained from him the barony of Little Dunmow, which being forfeited to the crown, in the year , by the felonious practices of William Baynard, was given by Henry to Gilbert, earl of Clare, and his heirs, together with the honours of Baynard castle. From him it descended in the female line to Robert Fitzwalter, who was castellan and banner bearer of London.

About , there arose a great contention betwixt king John and his barons on account of Matilda, called The Fair, a daughter of the above mentioned Robert Fitz-Walter, whom the

king unlawfully loved, but could not obtain :

In the year , king John being then in France with a great army, a truce was made between the kings of England and France for years. There being a river or arm of the sea between the armies, a knight among the English called out to those on the other side, to challenge any among them to come and take a just or with him, whereupon, without any delay, Robert Fitz-Walter, who was on the French side, ferried over, and got on horseback, without any to help him, and shewed himself ready to the face of this challenger, and at the course struck him so violently with his great spear, that both man and horse fell to the ground, and, when his spear was broken, he went back again to the king of France. King John, seeing this, cried out,

by God's tooth,

he were a king indeed who had such a knight.

O king, he is your knight; it is Robert Fitz- Walter.

This Robert died in the year , and was buried at Dunmow, and Walter his son succeeded him. This barony of Baynard was in the ward of king Henry during the non-age of another Robert Fitz-Walter, who, in the year , married Aelianor, daughter and heiress to the earl of Ferrers.

The rights that belonged to Robert Fitz-Walter, as castellan and banner-bearer of London, lord of Wodeham, were these:--

The said Robert and his heirs ought to be and are chief bannerets of London, in fee for the castelary, which he and his ancestors had by Castle Baynard in the said city. In time of war the said Robert and his heirs ought to serve the city in manner as followeth: that is,

The said Robert ought to come, he being the man of arms, on horseback, covered with cloth or armour, unto the great west door of , with his banner displayed before him of his arms. And, when he is come to the said door, mounted and apparelled as before is said, the mayor, with his aldermen and sheriffs, armed in their arms, shall come out of the said church

sir mayor, I am come to do my service which I owe to the city.

And the mayor and aldermen shall answer,

We give to you, as to our banneret of fee in this city, the banner of this city, to bear and govern to the honour of this city to your power.

And the said Robert and his heirs shall receive the banner in his hands, and go on foot out of the gate with the banner in his hands; and the mayor, aldermen, and sheriffs, shall follow to the door, and shall bring an horse to the said Robert, worth 20l. which horse shall be saddled with a saddle of the arms of the said Robert, and shall be covered with sindals of the said arms.

Also they shall present to him 20l. sterling, and deliver it to the chamberlain of the said Robert, for his expenses that day. Then the said Robert shall mount upon the horse which the mayor presented to him, with the banner in his hand; and, as soon as he is up, he shall say to the mayor, that he must cause a marshal to be chosen for the host, one of the city; which being done, the said Robert shall command the mayor and burgesses of the city to warn the commons to assemble, and all go under the banner of St. Paul; and the said Robert shall bear it himself to Aldgate, and there the said Robert and mayor shall deliver the said banner of St. Paul to whom they think proper. And, if they are to go out of the city, then the said Robert ought to chuse two out of every ward, the most sage persons, to look to the keeping of the city after they are gone out. And this counsel shall be taken in the priory of the Trinity near Aldgate. And before every town or castle which the host of London shall besiege, if the siege continue a whole year, the said Robert shall have, for every siege, of the commonalty of London, one hundred shillings, and no more.

These were the rights that Robert Fitz Walter had in time of war. The rights belonging to him and his heirs in the city of London, in time of peace, were as follow:

That is to say, the said Robert Fitz Walter had a soke or ward in the city, where was a wall of the canonry of St. Paul, which led down, by a brewhouse of St. Paul, to the Thames, and so to the side of the mill which was in the water coming down from Fleet-bridge, and went by London-wall betwixt the friars-preachers and Ludgate, and so returned by the house of the said friars to the wall of the canonry of St. Paul, that is, all the parish of St. Andrew, which was in the gift of his ancestors by the said seniority; and so the said Robert had appendant unto the said soke all the things underwritten:

That he ought to have a sokeman, and to place what sokeman he will, so he be of the sokemanry, or the same ward: and if any of the sokemanry be impleaded in the Guildhall of any thing that toucheth not the body of the mayor that for the time is, or that toucheth the body of no sheriff, it is not lawful for the sokeman of the sokemanry of the said Robert Fitz Walter to demand a court of the said Robert; and the mayor and his citizens of London ought to grant him to have a court; and in his court he ought to bring his judgments, as it is assented and agreed upon in the Guildhall, that shall be given him.

If any therefore be taken in his sokemanry, he ought to have his stocks and imprisonment in his soken; and he shall be brought from thence to the Guildhall before the mayor, and there they shall provide him his judgment that ought to be given of him; but his judgment shall not be published till he come into the court of the said Robert, and in his liberty.

And the judgment shall be such, that, if he have deserved death by treason, he to be tied to a post in the Thames, at a good wharf, where boats are fastened, two ebbings and two flowings of the water.

And if he be condemned for a common thief, he ought to be led to the elms, and there suffer his judgment as other thieves. And so the said Robert and his heirs hath honour, that he holdeth a great franchise within the city, and citizens are bound to do him right: that is to say, that, when the mayor will hold a great council, he ought to call the said Robert and his heirs to be with him in council of the city; and the said Robert ought to be sworn to be of council with the city against all people, saving the king and his heirs. And when the said Robert cometh to the hustings of the said Guildhall of the city. the mayor, or his lieutenant, ought to rise against him, and set him down near unto him; and, so long as he is in the Guildhall, all the judgments ought to be given by his mouth, according to the record of the recorders of the said Guildhall: and so many waifes as come so long as he is there, he ought to give them to his bailiffs of the town, or to whom he will, by the council of the mayor of the city.

The old castle was destroyed by fire, in , after which it was rebuilt by Humphrey, duke of Gloucester. At his decease, Henry VI. gave it to Richard, duke of York, who resided here, with his armed followers, to the number of , during the important convention of the great men of the nation, in , the forerunner of the civil wars between the houses of York and Lancaster.

This was also the residence of Richard III. when he took upon him the title of king. It was afterwards beautified, and made more commodious, by Hen. VII. who frequently lodged here. An act of parliament was passed in ( Henry VIII) for settling this castle on Henry duke of Richmond. The privy council met here, on the , for the purpose of proclaiming queen Mary; at which time it was the property and residence of William Herbert, earl of Pembroke.