I tried to calculate the side of an equilateral triangle inscribed in a square in the general case. I refer to your figure, assume BD=a, BG=d, BH=x and DI=y. Pythagore theorem applied to GH, GI and HI as hypothenuses gives the two equations :

x^2+d^2 = (a-d)^2+y^2 (1)

x^2+d^2 = a^2+(y-x)^2 (2)

from which I should calculate x and y when GHI is equilateral. I thought that was imposssible, which explains why I found no refernce to this problem on Internet. Later I took a different approach to the problem and found very easily the solution

x= (2*a-d)/sqr(3) and y= (a+d)/sqr(3)

I wonder how we can get these values directly from equations (1) and (2)