Let the center of the larger circle be (10,10)....and this is the incenter of the large triangle
Let A be the angle at the bottom right vertex
The tangent of this angle is 4/3
But the bisector of angle A creates an angle with 1/2 the measure of A
And the tangent of this angle is sqrt [ 1 -cos A] / sqrt [1 + cos A]
And cos A = 3/5
So tan (A/2) = sqrt [ 1-3/5 ] / sqrt [1 + 3/5] = 1/2
And
tan (A/2) =10/20
cos (A/2) = 2/sqrt (5)
Therefore, the distance from the bottom right vertex of the large triangle to the center of the larger circle can be found as
cos (A/2) = 20 / D
D = 20 / (2 /sqrt (5) ) = 10sqrt (5)
And using reflexive symmetry of similar polygons, the angle bisector will go through the center of both circles....
So using similar triangles we have that
10/ (10sqrt (5) ) = r / [ 10sqrt (5) - 10 - r ]
1/sqrt (5) = r / [10sqrt (5) -10 - r ] =
r sqrt (5) = 10sqrt (5) - 10 - r
r ( sqrt (5) + 1 ) = 10 ( sqrt (5) - 1)
r = 10 ( sqrt (5) -1) ( sqrt (5) -1) / 4
r = 10 ( 5 - 2sqrt (5) + 1) /4
r = 15 - 5sqrt (5)
Here's a pic :