Stitching tweak walkthrough

Tweaking BakingTray and StitchIt for high stitching accuracy

This page is a walkthrough explaining how to improving stitching accuracy using BakingTray and StitchIt. It's description of a real session during which a mistake a was made that then needed correction.

First look

We begin by stitching a section and looking at it:

>> IM=chessboardStitch([75,1],2); %Slow for larger datasets 
>> exploreChessboard(IM,4000) %Slow for larger datasets

The fact the red tile is above the green on one side and below on the other tells me there is a rotation problem.

The bidirectional scanning correction in ScanImage wasn’t set quite right. We can fix this post- hoc, although the correction is a bit hackish at the moment.

Indeed, upon zooming in to an area with obvious features the microns per pixel is very far off:

Indeed, microns per pixel is very far off:

First tweak

What are the values?

>> IM.params.voxelSize
ans =
struct with fields:
X: 0.8060 Y: 0.8060 Z: 5

We know it needs to be a larger number so the tiles end up overlapping more. Let’s try just one axis first:

>> IM(2)=IM; %copy
>> IM(2).params.voxelSize.Y=0.82; %New value
>> IM(2)=chessboardStitch([75,1],2,IM(2).params); %re-stitch
>> exploreChessboard(IM,4000) %Compare new and previous versions

Note that now IM has a length of 2. When we run exploreChessboard with IM having a length of 2, we see the two images side by side:

And if we zoom in:

Adding a tile rotation

It looks much closer. They are offset because of the rotation we noted earlier, but we’ll fix that later. Let’s do the other axis too.

>> IM(2).params.voxelSize.X=0.815; %New value
>> IM(2)=chessboardStitch([75,1],2,IM(2).params); %re-stitch
>> exploreChessboard(IM,4000) %Compare new and original versions

Closer. But’ll play with rotation before going back to mics per pixel. [NOTE: AT THIS POINT I WAS ASSUMING THAT THE GREEN FIBRE AT THE UPPER OVERLAP REGION IS PART OF THE RED BELOW IT. TURNS OUT LATER THAT IT WASN’T, BUT I DIDN’T REALISE THAT HERE]

To make this easier I will use affineMatGen that is in BakingTray. I don’t have this function in my path on my desktop PC so I add it now:

>> ls ~/work/Anatomy/code/bakingtray/code/BTresources/
LimitFigSize.m TileStepSize.m plotboxpos.m previewFilesToStack.m NumTiles.m affineMatGen.m prettyTime.m

>> addpath('~/work/Anatomy/code/bakingtray/code/BTresources/')

Create a matrix with a small rotation and add it to IM(2).params:

>> affineMatGen('rot',-0.3)
- [0.999986, -0.005236, 0.0] - [0.005236, 0.999986, 0.0] - [0.0, 0.0, 1.0]
>> A=affineMatGen('rot',-0.3) A=
1.0000 -0.0052 0 0.0052 1.0000 0 0 0 1.0000
>> IM(2).params.affineMat ans =
[]
>> IM(2).params.affineMat=A;

Let’s see what that does. Stitch and explore as before. In the area we were looking at before, stuff is better:

But I really care about the sample edges right now, as rotations are really obvious there. Especially along the dorsal and ventral surfaces. I therefore re-explore with a lower threshold and look there.

>> exploreChessboard(IM,2000)

We can do better. There is still rotation and it looks like we didn’t over-compensate. Let’s start fine-tweaking by comparing the above, nicer, version to a new version:

>> IM(3)=IM(2); %Copy again
>> A=affineMatGen('rot',-0.35);
>> IM(3).params.affineMat=A;
>> IM(3)=chessboardStitch([75,1],2,IM(3).params);
>> exploreChessboard(IM(2:3),2000)

That wasn’t enough [NOT SHOWN] so repeat with more rotation:

>> A=affineMatGen('rot',-0.45);
>> IM(3).params.affineMat=A;
>> IM(3)=chessboardStitch([75,1],2,IM(3).params); exploreChessboard(IM(2:3),2000)

That still wasn’t enough [NOT SHOWN] so repeat with more rotation:

>> IM(3).params.affineMat=affineMatGen('rot',-0.5);
>> IM(3)=chessboardStitch([75,1],2,IM(3).params); exploreChessboard(IM(2:3),2000)

Finally this is much better:

Certainly not perfect. Never is perfect. But very much better. Try another 0.05 degrees and compare what we just did to that:

>> IM(2)=IM(3); %Replace second with third to get a yet finer comparison
>> IM(3).params.affineMat=affineMatGen('rot',-0.55); IM(3)=chessboardStitch([75,1], 2,IM(3).params); exploreChessboard(IM(2:3),2000)

This is again slightly better [NOT SHOWN]. We can leave the rotation for now and go back and explore the dataset more carefully, considering microns per pixel again.

>> exploreChessboard(IM(2:3),4000) %Change threshold to higher value again

The region we first concentrated on looks better than before the rotation correction:

Exploring Y microns/pixel

But maybe microns per pixel in Y still needs tweaking? We need to pan around the sample to see what’s going on elsewhere. Can’t base everything on just one area. I see this, which I don’t like. This is miles out and can only be explained by stage motion errors, or the sample shifting, or a tilted focal plane, or the X value still being wrong. Tilted focal plane is unlikely to have such a big effect, though. [NOTE: THIS IS WHEN I STARTED TO QUESTION WHAT THAT FIBRE WAS AT THE START]

I also see this, which looks very much like tilted focal plane, but that’s an error along the other scan axis.

Let’s go with my X value being wrong. Let’s look around for other evidence of this.

Looking at the EM grid image for this system

Yes, it looks like that’s what’s happening. Note that I’m paying attention to regions near the middle of the tiles. Looking at the grid image quickly in Fiji for this microscope I see this:

I had to rotate tiles 90 degrees to stitch, so the left and right edges are the ones I care about. Note there is pincushion distortion there. The other two axes seem good. Let’s correct the grid image first. The X value was set based upon pixels near a corner, so we’ll not touch it yet. I also note that this grid image has a bidirectional scanning artefact too.

Let’s have a fiddle:

>> imagesc(G(:,:,1)) %Loaded into MATLAB now
>> caxis([0,5000]), axis square

%Play a bit until we get this
>> imagesc(stitchit.tools.lensdistort(G(:,:,1),[0.04,0.0]))

>> IM(3).params.lensDistort.cols=rows; %We want to work along the rows
>> IM(3)=chessboardStitch([75,1],2,IM(3).params); exploreChessboard(IM(2:3),2000)

This looks like we’ve over-compensated, but we know that the grid image looks square.

Go back and tweak X

So we need to tweak X. This is why what I should have done was fix the barrel distortion before doing anything else.

>> IM(3).params.voxelSize
X: 0.8150 Y: 0.8200 Z: 5
>> IM(3).params.voxelSize.X=0.8;
>> IM(3)=chessboardStitch([75,1],2,IM(3).params); exploreChessboard(IM(2:3),2000)

Iterate a bit until stuff looks good. We cycle through:

>> IM(3).params.voxelSize.X=0.805;
>> IM(3).params.voxelSize.X=0.8075;

And now we see:

That stuff that was previously bad is now good. Pan around a bit. It all looks about as good as expected considering the tilt in the focal plane. This can only be corrected by tilting the stage. The only other option would be affine transformation in Z, but that would require very dense sampling in Z which we don’t typically do.

Commit to recipe file

In this case we want to tweak an already acquired sample so we commit our changes to the recipe file:

It would also be a good idea to add these changes to the frameSizes settings file in BakingTray so all future samples have these good parameters saved with them.

% stitching from this file produces the good stuff
>>IMfinal=chessboardStitch([75,1],2);
>> exploreChessboard(IMfinal,4000)